From voytek at sbt.net.au Sun Jul 1 04:37:58 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Sun, 1 Jul 2012 11:37:58 +1000 Subject: [Dovecot] ot: trying to understand log entries, etc Message-ID: totally of, I'm trying to understand what's happening, aka, 'how does email works' ? I have Dovecot 1.x, all works fine I noticed in the logs failed access from no-longer-hosted account/user, physical account deleted/removed, most likely from user's BBerry, every ~15 minutes: from /var/log/maillog --- Jun 24 07:31:25 dovecot: imap-login: Disconnected: Inactivity (auth failed, 1 attempts): user=, method=PLAIN, rip=101.222.222.222, lip=111.111.111.111 --- I recreated account/physical path /var/mail/.../tld/vvv at tld I logged in with webmail, all works, webmail logged out I physically copied some emails (from another account) to physical path ../../tld/vvv at tld/new after a while, I see the email files NO LONGER in ../../../new BUT now in ../../tld/vvv at tld/cur Q: that means the email was accessed, doesn't it ? BUT, I don't see any more access in the /var/log/maillog, not failed, not OK I did see one, after I recreated account Jun 30 20:01:53 dovecot: IMAP(vvv at tld): Disconnected: Logged out bytes=32/666 NONE since, but email files move from new to cur ?? if I observe email files moving from 'new' to 'cur' shouldn't I see some action logged in /var/log/maillog to correspond ? perplexed.... -- Voytek From stan at hardwarefreak.com Sun Jul 1 10:17:35 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 01 Jul 2012 02:17:35 -0500 Subject: [Dovecot] RAID1+md concat+XFS as mailstorage In-Reply-To: <213B51F00051AE48A9F0E11288017717B84022@Delta.sc.local> References: <213B51F00051AE48A9F0E11288017717B8401D@Delta.sc.local><4FEC4ACD.20104@wildgooses.com> <4FEE8D6E.3030703@hardwarefreak.com> <213B51F00051AE48A9F0E11288017717B84022@Delta.sc.local> Message-ID: <4FEFF98F.4020409@hardwarefreak.com> On 6/30/2012 6:17 AM, ???????? ????????? ?????????? wrote: > So, you say that one should use this configuration in production with > hope that such failure would never happen? No, I'm saying you are trolling. A concat of RAID1 pairs has reliability identical to RAID10. I don't see you ripping a mirror pair from a RAID10 array and saying RAID10 sucks. Your argument has several flaws. In a production environment, a dead drive will be replaced and rebuilt before the partner fails. In a production environment, the mirror pairs will be duplexed across two SAS/SATA controllers. Duplexing the mirrors makes a concat/RAID1, and a properly configured RAID10, inherently more reliable than RAID5 or RAID6, which simply can't be protected against controller failure. By stating the concat/RAID1 configuration is unreliable simply shows your ignorance of storage system design and operation. -- Stan > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of Stan Hoeppner > Sent: Saturday, June 30, 2012 4:24 PM > To: dovecot at dovecot.org > Subject: Re: [Dovecot] RAID1+md concat+XFS as mailstorage > > On 6/28/2012 7:15 AM, Ed W wrote: >> On 28/06/2012 13:01, ???????? ????????? ?????????? wrote: > >>> somewhere in maillist I've seen RAID1+md concat+XFS being promoted as >>> mailstorage. >>> Does anybody in here actually use this setup? >>> >>> I've decided to give it a try, >>> but ended up with not being able to recover any data off survived >>> pairs from linear array when _the_first of raid1 pairs got down. > > The failure of the RAID1 pair was due to an intentional breakage test. > Your testing methodology was severely flawed. The result is the correct > expected behavior of your test methodology. Proper testing will yield a > different result. > > One should not be surprised that something breaks when he intentionally > attempts to break it. > >> This is the configuration endorsed by Stan Hoeppner. > > Yes. It works very well for metadata heavy workloads, i.e. maildir. > From CMarcus at Media-Brokers.com Sun Jul 1 13:34:14 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 01 Jul 2012 06:34:14 -0400 Subject: [Dovecot] RAID1+md concat+XFS as mailstorage In-Reply-To: <4FEDD2DC.7080404@wildgooses.com> References: <213B51F00051AE48A9F0E11288017717B8401D@Delta.sc.local><4FEC4ACD.20104@wildgooses.com><213B51F00051AE48A9F0E11288017717B8401E@Delta.sc.local> <4FEC548E.4030405@wildgooses.com> <213B51F00051AE48A9F0E11288017717B8401F@Delta.sc.local> <4FEC844A.9090302@wildgooses.com> <4FEC8C4E.1020209@Media-Brokers.com> <4FECC01C.90303@wildgooses.com> <4FED8E38.1020105@Media-Brokers.com> <4FEDD2DC.7080404@wildgooses.com> Message-ID: <4FF027A6.1020300@Media-Brokers.com> On 2012-06-29 12:07 PM, Ed W wrote: > On 29/06/2012 12:15, Charles Marcus wrote: >> Depends on what you mean exactly by 'incorrect'... > I'm sorry, this wasn't meant to be an attack on you, No worries - it wasn't taken that way - I simply disagreed with the main point you were making, and still do. While I do agree there is some truth to the issue you have raised, I just don't see it as quite the disaster-in-waiting that you do. I have been running small RAID setups for quite a while, and while I had one older RAID5 (with NO hot spare) that I inherited (this was many years ago) that gave me fits for about a month once (had drives randomly 'failing', but a rebuild - which took a few HOURS, and this was with small (by today's standards - 120GB drives) would fix it, then another one would do drop out 2 or 3 days later, etc. I finally found an identical replacement controller on ebay (old 3ware card) and once it was replaced it fixed the problem). I also had one instance in a RAID10 setup I configured myself a few years ago where one of the pairs had some errors on an unclean shutdown (this was after about 3 years of 24/7 operation on a mail server) and went into automatic rebuild, which went smoothly (and was mucho faster than the RAID5 rebuilds were even though the drives were much bigger). So, yes, while I acknowledge the risk, it is the risk we all run storing data on hard drives. > I thought I was pointing out what is now fairly obvious stuff, but > it's only recently that the maths has been popularised by the common > blogs on the interwebs. Whilst I guess not everyone read the flurry > of blog articles about this last year, I think it's due to be > repeated in increasing frequency as we go forward: > > The most recent article which prompted all of the above is I think this > one: > http://queue.acm.org/detail.cfm?id=1670144 > More here (BARF = Battle Against Raid 5/4) > http://www.miracleas.com/BAARF/ I'll find time to read these over the next week or two, thanks... > Intel have a whitepaper which says: > > Intelligent RAID 6 Theory Overview And Implementation > > RAID 5 systems are commonly deployed for data protection in most > business environments. While maybe true many years ago, I don't think this is true today. I wouldn't touch RAID5 with a ten foot pole, but yes, maybe there are still people who use it for some reason - and maybe there are some corner cases where it is even desirable? > However, RAID 5 systems only tolerate a single drive failure, and the > probability of encountering latent defects [i.e. UREs, among other > problems] of drives approaches 100 percent as disk capacity and array > width increase. Well, this is definitely true, but I wouldn't touch RAID5 today. > And to be clear - RAID5/RAID1 has a very significant probability that > once your first disk has failed, in the process of replacing that disk > you will discover an unrecoverable error on your remaining drive and > hence you have lost some data... Well, this is true, but the part of your comment that I was responding to and challenging was that the entire RAID just 'died' and you lost ALL of your data. That is simply not true on modern systems. >>> So the vulnerability is not the first failed disk, but discovering >>> subsequent problems during the rebuild. >> True, but this applies to every RAID mode (RAID6 included). > No, see RAID6 has a dramatically lower chance of this happening than > RAID1/5. See this is the real insight and I think it's important that > this fairly (obvious in retrospect) idea becomes widely known and > understood to those who manage arrays. > RAID6 needs a failed drive and *two* subsequent errors *per stripe* to > lose data. RAID5/1 simply need one subsequent error *per array* to lose > data. Quite a large difference! Interesting... I'll look at this more closely then, thanks. >> Also, one big disadvantage of RAID5/6 is the rebuild times > Hmm, at least theoretically both need a full linear read of the other > disks. The time for an idle array should be similar in both cases. Agree > though that for an active array the raid5/6 generally causes more drives > to read/write, hence yes, the impact is probably greater. No 'probably' to it. It is definitely greater, even comparing the smallest possible RAID setups (4 drives are minimum for each). But, as the size of (number of disks in) the array increases, the difference increases dramatically. With RAID10, when a drive fails and a rebuild occurs, only ONE drive must be read (remirrored) - in a RAID5/6, most if not *all* of the drives must be read from (depends on how it is configured I guess). > However, don't miss the big picture, your risk is a second error > occurring anywhere on the array with raid1/5, but with raid 6 your risk > is *two* errors per stripe, ie you can fail a whole second drive and > still continue rebuilding with raid6 And is the same with a RAID10, as long as the second drive failure isn't the one currently being remirrored. I think you have proven your case that a RAID6 is statistically a little less likely to suffer a catastrophic cascading disk failure scenario than RAID10. > I personally feel that raid arrays *are* very fragile. Backups are often > the option when you get multi-drive failures (even if theoretically the > array is repairable). However, it's about the best option we have right > now, so all we can do is be aware of the limitations... And since backups are stored on drives (well, mine are, I stopped using tape long ago), they have the same associated risks... but of course I agree with you that they are absolutely essential. > Additionally I have very much suffered this situation of a failing RAID5 > which was somehow hanging together with just the odd uncorrectable read > error reported here and there (once a month say). I copied off all the > data and then as an experiment replaced one disk in this otherwise > working array, which then triggered a cascade of discovered errors all > over the disk and rebuilding was basically impossible. Sounds like you had a bad controller to me... and yes, when a controller goes bad, lots of weirdness and 'very bad things' can occur. > Roll on btrfs I say... +1000 ;) -- Best regards, Charles From CMarcus at Media-Brokers.com Sun Jul 1 13:48:04 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 01 Jul 2012 06:48:04 -0400 Subject: [Dovecot] RAID1+md concat+XFS as mailstorage In-Reply-To: <4FEFF98F.4020409@hardwarefreak.com> References: <213B51F00051AE48A9F0E11288017717B8401D@Delta.sc.local><4FEC4ACD.20104@wildgooses.com> <4FEE8D6E.3030703@hardwarefreak.com> <213B51F00051AE48A9F0E11288017717B84022@Delta.sc.local> <4FEFF98F.4020409@hardwarefreak.com> Message-ID: <4FF02AE4.4080900@Media-Brokers.com> On 2012-07-01 3:17 AM, Stan Hoeppner wrote: > In a production environment, the mirror pairs will be duplexed across > two SAS/SATA controllers. > > Duplexing the mirrors makes a concat/RAID1, and a properly configured > RAID10, inherently more reliable than RAID5 or RAID6, which simply can't > be protected against controller failure. Stan, am I correct that this - dual/redundant controllers - is the reason that a real SAN is more reliable than just running local storage on a mod-high end server? -- Best regards, Charles From bentley at crenelle.com Sun Jul 1 23:04:30 2012 From: bentley at crenelle.com (Michael Brian Bentley) Date: Sun, 01 Jul 2012 13:04:30 -0700 Subject: [Dovecot] Config off by a nuance or a gross? Message-ID: <4FF0AD4E.7080800@crenelle.com> Hi, I am trying to establish an IMAP mail service accessible using a current Thunderbird on a laptop. I used Macports to install Dovecot2 (2.1.5) on an older Snow Leopard Mac mini with Intel Core Duo (specifically a Macmini1,1). My goal is to be able to run sieve under dovecot2 and offload the email triage from my work laptop to the little mail server. Mail appears to show up on the server just fine. I'm having trouble getting Tbird to log in and access properly. Because there are so many components in play (Tbird, pam, Dovecot2, macports, OS X Snow Leopard 10.6.8) (dovecot by itself seems to have quite a few settings), it is hard to tell what bit is out of whack. The message I get from Tbird 13.0.1 for OS X (on Lion) is: Alert: The IMAP server bentley on TheMini does not support the selected authentication method. Please change the 'Authentication method' in the 'Account Settings | Server settings'. When I set up the account, Tbird's automatic configuration invents a fake mail service based on my domain name, but appears to configure things sensibly once I switch to the manual configuration and fill in the relevant IP information. It does not appear to care whether it has a password or not. When I set up the account manually on TBird, I let it set up as: IMAP (ipv4)xx.xxx.xx.xx Port: 143 Connection security: None Authentication method: Password, transmitted insecurely SMTP xxxxxxxx.com Port: 587 The dovecot.conf tree includes these: passdb { driver = pam } userdb { driver = passwd } default_login_user = _dovenull default_internal_user = _dovecot service imap-login { executable = /opt/local/libexec/dovecot/imap-login inet_listener imap { address = * port = 143 } } service imap { executable=/opt/local/libexec/dovecot/imap process_limit = 1024 } service lmtp { unix_listener lmtp { } } Issuing the command: dovecot --build-options produces: Build options: ioloop=kqueue notify=kqueue ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw SQL drivers: Passdb: checkpassword pam passwd passwd-file Userdb: checkpassword passwd prefetch passwd-file The /etc/pam.d/dovecot file contains: # pop3, imap, smtp : auth account password session auth required pam_opendirectory.so account required pam_permit.so password required pam_deny.so session required pam_permit.so Dovecot provides the following debug messages on the console when I add args = failure_show_msg=yes to passdb in the relevant dovecot conf file: Jul 1 11:12:38 xx-xxx-xx-xx-nerewhon dovecot[3871]: auth: Debug:auth client connected (pid=5047) Jul 1 11:12:40 xx-xxx-xx-xx-nerewhon dovecot[3871]: imap-login: Disconnected (no auth attempts in 2 secs): user=<>, rip=192.168.xx.xx, lip=xx.xxx.xx.xx I recently discovered doveadm and used it with: doveadm config to create what I think is the current dovecot.conf file. I hesitate to inflict that on the list without being told, "yeah, we need that," there may be digest subscribers out there... I also see these messages show up; they appear unrelated to what I do with Tbird: Jul 1 11:37:33 xx-xxx-xx-xx-nerewhon dovecot[3871]: auth: Debug: Loading modules from directory: /opt/local/lib/dovecot/auth auth: Debug: auth client connected (pid=5094) auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49774 auth: Debug: client out: CONT 1 auth: Debug: client in: CONT 1 AGJlbnRsZXkAZ3IxbmNo auth-worker(5100): Debug: Loading modules from directory: /opt/local/lib/dovecot/auth auth-worker(5100): Debug: pam(bentley,127.0.0.1): lookup service=dovecot auth-worker(5100): Debug: pam(bentley,127.0.0.1): #1/1 style=1 msg=Password: auth-worker(5100): pam(bentley,127.0.0.1): pam_authenticate() failed: authentication error (password mismatch?) (given password: whatever) auth: Debug: client out: FAIL 1 user=bentley reason=Password: auth: Debug: client in: AUTH 2 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49774 resp=AGJlbnRsZXkAZ3IxbmNo auth-worker(5100): Debug: pam(bentley,127.0.0.1): lookup service=dovecot auth-worker(5100): Debug: pam(bentley,127.0.0.1): #1/1 style=1 msg=Password: auth-worker(5100): pam(bentley,127.0.0.1): pam_authenticate() failed: authentication error (password mismatch?) (given password: whatever) auth: Debug: client out: FAIL 2 user=bentley reason=Password: imap-login: Disconnected (auth failed, 2 attempts in 9 secs): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured auth: Debug: Loading modules from directory: /opt/local/lib/dovecot/auth From stan at hardwarefreak.com Mon Jul 2 02:12:12 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 01 Jul 2012 18:12:12 -0500 Subject: [Dovecot] RAID1+md concat+XFS as mailstorage In-Reply-To: <4FF02AE4.4080900@Media-Brokers.com> References: <213B51F00051AE48A9F0E11288017717B8401D@Delta.sc.local><4FEC4ACD.20104@wildgooses.com> <4FEE8D6E.3030703@hardwarefreak.com> <213B51F00051AE48A9F0E11288017717B84022@Delta.sc.local> <4FEFF98F.4020409@hardwarefreak.com> <4FF02AE4.4080900@Media-Brokers.com> Message-ID: <4FF0D94C.9000609@hardwarefreak.com> On 7/1/2012 5:48 AM, Charles Marcus wrote: > On 2012-07-01 3:17 AM, Stan Hoeppner wrote: >> In a production environment, the mirror pairs will be duplexed across >> two SAS/SATA controllers. >> >> Duplexing the mirrors makes a concat/RAID1, and a properly configured >> RAID10, inherently more reliable than RAID5 or RAID6, which simply can't >> be protected against controller failure. > > Stan, am I correct that this - dual/redundant controllers - is the > reason that a real SAN is more reliable than just running local storage > on a mod-high end server? In this case I was simply referring to using two PCIe SAS HBAs in a server, mirroring drive pairs across the HBAs with md, then concatenating the RAID1 pairs with md --linear. This gives protection against all failure modes. You can achieve the former with RAID5/6 but not the latter. Consider something like: 2x http://www.lsi.com/products/storagecomponents/Pages/LSISAS9200-8e.aspx 2x http://www.dataonstorage.com/dataon-products/6g-sas-jbod/dns-1640-2u-24-bay-6g-25inch-sassata-jbod.html 48x Seagate ST9300605SS 300GB SAS 10k RPM This hardware yields a high IOPS, high concurrency, high performance mail store. Drives are mirrored across HBAs and JBODs. Each HBA is connected to an expander/controller in both chassis, yielding full path redundancy. Each controller can see every disk in both enclosures. With this setup and SCSI multipath, you have redundancy against drive, HBA, cable, expander, and chassis failure. You can't get any more redundant than that. And you can't achieve this with RAID5/6, unless you simply mirror two of them across the HBAs/chassis. But that eliminates the whole reason for RAID5/6--space/cost efficiency. People like ???????? ????????? ?????????? may wish to waste 3 more HBAs, JBOD chassis, and 24 more drives for a 3-way md mirror, to protect against the 1000 year scenario of two drives in the same mirror pair failing before the first is rebuilt. A quality SAN head with dual redundant controllers will give you all of the above for all RAID levels, assuming you have multiple network connections (FC or iSCSI). You'd need two HBAs in the host, one connected to each controller. This is a direct connect scenario. In a fabric scenario, you'd have an independent FC or iSCSI switch on each path. And of course you need SCSI multipath configured on the host. -- Stan From amateo at um.es Mon Jul 2 09:49:23 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 02 Jul 2012 08:49:23 +0200 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply In-Reply-To: <20120629203334.GA7718@daniel.localdomain> References: <20120428110023.GA9236@daniel.localdomain> <1335833212.21461.82.camel@innu> <20120629203334.GA7718@daniel.localdomain> Message-ID: <4FF14473.4020002@um.es> El 29/06/12 22:33, Daniel Parthey escribi?: > Timo Sirainen wrote: >> On Sat, 2012-04-28 at 13:00 +0200, Daniel Parthey wrote: >> >>> we are experiencing similar sporadic data timeout issues with dovecot 2.0.20 >>> as in http://dovecot.org/pipermail/dovecot/2011-June/059807.html >>> at least once a week. Some mails get temporarily deferred in the >>> postfix queue since dovecot director lmtp refuses them and the >>> mails are delivered at a later time. >> >> What isn't in v2.0 is the larger rewrite of the LMTP proxying >> code in v2.1, which I hope fixes also this timeout problem. > > Same problem persists after update to 2.1.7, especially for distribution > lists which contain several target email addresses which are then > pipelined by postfix through a single lmtp proxy connection: > > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Jun 29 10:14:03 10.129.3.233 postfix/lmtp[29674]: 00318C090: to=, orig_to=, relay=127.0.0.1[127.0.0.1]:20024, delay=31, delays=1/0.16/0.01/30, dsn=4.4.0, status=deferred (host 127.0.0.1[127.0.0.1] said: 451 4.4.0 Remote server not answering (timeout while waiting for reply to DATA reply) (in reply to end of DATA command)) > Hello, I had a similar issue. This data timeout is reported for all recipients in the same lmtp connection? What is the start time of the lmtp connection? My problem was that this timeout seems to be counted from the beginning of the LMTP connection, so when I have a lot of recipients in the same connection, last ones sometimes timedout. I solved it increasing this timeout with proxy_timeout option and reducing max number of lmtp recipients in postfix. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From amateo at um.es Mon Jul 2 09:53:38 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 02 Jul 2012 08:53:38 +0200 Subject: [Dovecot] director directing to wrong server (sometimes) In-Reply-To: <20120630015150.GA12291@daniel.localdomain> References: <4FED8B42.5010701@um.es> <20120630015150.GA12291@daniel.localdomain> Message-ID: <4FF14572.5070102@um.es> El 30/06/12 03:51, Daniel Parthey escribi?: > Hi Angel, > > Angel L. Mateo wrote: >> I have a user, its assigned server is 155.54.211.164. The problem >> is that I don't know why director sent him yesterday to a different >> server, because my server was up all the time. Moreover, I'm using >> poolmon in director servers to check availability of final servers >> and it didn't report any problem with the server. > > Which version of dovecot are you using? > "doveconf -n" of director and mailbox instance? > Sorry. Here you have them > You should monitor the output of > doveadm director status username at example.org > doveadm director ring status > on each of the directors over time with a timestamp. > > This might shed some light on where the user is directed and why, > and ring status will tell which directors can see each other. > doveadm director move can also influence where a user is sent, > but this will be reflected by "Current:" entry of director status, > there you can also find the time when the entry in hashtable > will expire. > I have running poolmon. It didn't report any problem in its logs. I have also check all dovecot logs and I don't have any error. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 -------------- next part -------------- # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-24-generic x86_64 Ubuntu 12.04 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_username_format = %n auth_verbose = yes default_process_limit = 1000 director_mail_servers = 155.54.211.161-155.54.211.164 director_servers = 155.54.211.185 155.54.211.186 disable_plaintext_auth = no doveadm_proxy_port = 24245 lmtp_proxy = yes log_timestamp = %Y-%m-%d %H:%M:%S mail_debug = yes passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y proxy_timeout=150 driver = static } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 } service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap-login { client_limit = 2206 executable = imap-login director process_limit = 6 process_min_avail = 6 service_count = 0 } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3-login { client_limit = 833 executable = pop3-login director process_limit = 6 process_min_avail = 6 service_count = 0 } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } -------------- next part -------------- # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-24-generic x86_64 Ubuntu 12.04 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_username_format = %n auth_verbose = yes default_process_limit = 1000 director_mail_servers = 155.54.211.161-155.54.211.164 director_servers = 155.54.211.185 155.54.211.186 disable_plaintext_auth = no doveadm_proxy_port = 24245 lmtp_proxy = yes log_timestamp = %Y-%m-%d %H:%M:%S mail_debug = yes passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = proxy=y nopassword=y proxy_timeout=150 driver = static } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 } service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap-login { client_limit = 2206 executable = imap-login director process_limit = 6 process_min_avail = 6 service_count = 0 } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3-login { client_limit = 833 executable = pop3-login director process_limit = 6 process_min_avail = 6 service_count = 0 } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From tss at iki.fi Mon Jul 2 10:10:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 Jul 2012 10:10:10 +0300 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply In-Reply-To: <4FF14473.4020002@um.es> References: <20120428110023.GA9236@daniel.localdomain> <1335833212.21461.82.camel@innu> <20120629203334.GA7718@daniel.localdomain> <4FF14473.4020002@um.es> Message-ID: On 2.7.2012, at 9.49, Angel L. Mateo wrote: > My problem was that this timeout seems to be counted from the beginning of the LMTP connection, so when I have a lot of recipients in the same connection, last ones sometimes timedout. I solved it increasing this timeout with proxy_timeout option and reducing max number of lmtp recipients in postfix. Ah, interesting. These should help: http://hg.dovecot.org/dovecot-2.1/rev/27dccff46fe9 http://hg.dovecot.org/dovecot-2.1/rev/8a97daa8aff6 From tss at iki.fi Mon Jul 2 10:13:50 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 Jul 2012 10:13:50 +0300 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply In-Reply-To: References: <20120428110023.GA9236@daniel.localdomain> <1335833212.21461.82.camel@innu> <20120629203334.GA7718@daniel.localdomain> <4FF14473.4020002@um.es> Message-ID: <0B007B5E-E2E7-41B3-A62F-BBB5F886A0C2@iki.fi> On 2.7.2012, at 10.10, Timo Sirainen wrote: > On 2.7.2012, at 9.49, Angel L. Mateo wrote: > >> My problem was that this timeout seems to be counted from the beginning of the LMTP connection, so when I have a lot of recipients in the same connection, last ones sometimes timedout. I solved it increasing this timeout with proxy_timeout option and reducing max number of lmtp recipients in postfix. > > Ah, interesting. These should help: > > http://hg.dovecot.org/dovecot-2.1/rev/27dccff46fe9 > http://hg.dovecot.org/dovecot-2.1/rev/8a97daa8aff6 Plus http://hg.dovecot.org/dovecot-2.1/rev/569588ff7ef0 although I'm not entirely sure if it's needed. The LMTP code is rather ugly and difficult to follow.. From tss at iki.fi Mon Jul 2 10:17:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 Jul 2012 10:17:46 +0300 Subject: [Dovecot] director directing to wrong server (sometimes) In-Reply-To: <4FF14572.5070102@um.es> References: <4FED8B42.5010701@um.es> <20120630015150.GA12291@daniel.localdomain> <4FF14572.5070102@um.es> Message-ID: On 2.7.2012, at 9.53, Angel L. Mateo wrote: >>> I have a user, its assigned server is 155.54.211.164. The problem >>> is that I don't know why director sent him yesterday to a different >>> server, because my server was up all the time. Moreover, I'm using >>> poolmon in director servers to check availability of final servers >>> and it didn't report any problem with the server. >> >> Which version of dovecot are you using? >> "doveconf -n" of director and mailbox instance? >> > Sorry. Here you have them I think this shouldn't matter, but would be better to make both %n anyway: auth_username_format = %n director_username_hash = %u From ewald.lists at fun.de Mon Jul 2 12:07:04 2012 From: ewald.lists at fun.de (Ewald Dieterich) Date: Mon, 02 Jul 2012 11:07:04 +0200 Subject: [Dovecot] UTF-8 mailbox name STATUS response Message-ID: <4FF164B8.7000107@fun.de> STATUS in Dovecot 2.1.7 returns the UTF-8 decoded folder name in a string literal: . CREATE "INBOX.Euro &IKw-" . OK Create completed. . LIST "" "INBOX.Euro &IKw-" * LIST (\HasNoChildren) "." "INBOX.Euro &IKw-" . STATUS "INBOX.Euro &IKw-" (MESSAGES) * STATUS {14} INBOX.Euro ? (MESSAGES 0) . OK Status completed. Is this intended? My Perl script based on Mail::IMAPTalk doesn't like it and I wonder if there are other clients that don't cope with it. From tss at iki.fi Mon Jul 2 12:25:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 Jul 2012 12:25:44 +0300 Subject: [Dovecot] UTF-8 mailbox name STATUS response In-Reply-To: <4FF164B8.7000107@fun.de> References: <4FF164B8.7000107@fun.de> Message-ID: <7A8C8BD1-3C7E-4E7B-BCCB-9A6CBD7C698C@iki.fi> On 2.7.2012, at 12.07, Ewald Dieterich wrote: > STATUS in Dovecot 2.1.7 returns the UTF-8 decoded folder name in a > string literal: > > . CREATE "INBOX.Euro &IKw-" > . OK Create completed. > . LIST "" "INBOX.Euro &IKw-" > * LIST (\HasNoChildren) "." "INBOX.Euro &IKw-" > . STATUS "INBOX.Euro &IKw-" (MESSAGES) > * STATUS {14} > INBOX.Euro ? (MESSAGES 0) > . OK Status completed. > > Is this intended? My Perl script based on Mail::IMAPTalk doesn't like > it and I wonder if there are other clients that don't cope with it. Most likely that mailbox exists like that in the filesystem. It shouldn't. Dovecot hasn't allowed creating those for several years now. There's no good way to handle that, because even if Dovecot translated it to mUTF-7 it couldn't access the mailbox because it wouldn't exist as mUTF-7 in the filesystem.. From amateo at um.es Mon Jul 2 12:31:22 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 02 Jul 2012 11:31:22 +0200 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply In-Reply-To: <0B007B5E-E2E7-41B3-A62F-BBB5F886A0C2@iki.fi> References: <20120428110023.GA9236@daniel.localdomain> <1335833212.21461.82.camel@innu> <20120629203334.GA7718@daniel.localdomain> <4FF14473.4020002@um.es> <0B007B5E-E2E7-41B3-A62F-BBB5F886A0C2@iki.fi> Message-ID: <4FF16A6A.8070209@um.es> El 02/07/12 09:13, Timo Sirainen escribi?: > On 2.7.2012, at 10.10, Timo Sirainen wrote: > >> On 2.7.2012, at 9.49, Angel L. Mateo wrote: >> >>> My problem was that this timeout seems to be counted from the beginning of the LMTP connection, so when I have a lot of recipients in the same connection, last ones sometimes timedout. I solved it increasing this timeout with proxy_timeout option and reducing max number of lmtp recipients in postfix. >> >> Ah, interesting. These should help: >> >> http://hg.dovecot.org/dovecot-2.1/rev/27dccff46fe9 >> http://hg.dovecot.org/dovecot-2.1/rev/8a97daa8aff6 > > Plus http://hg.dovecot.org/dovecot-2.1/rev/569588ff7ef0 although I'm not entirely sure if it's needed. The LMTP code is rather ugly and difficult to follow.. > Are these patches included in any released? (I have 2.1.5) -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tss at iki.fi Mon Jul 2 12:51:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 Jul 2012 12:51:41 +0300 Subject: [Dovecot] lmtp proxy timeout while waiting for reply to DATA reply In-Reply-To: <4FF16A6A.8070209@um.es> References: <20120428110023.GA9236@daniel.localdomain> <1335833212.21461.82.camel@innu> <20120629203334.GA7718@daniel.localdomain> <4FF14473.4020002@um.es> <0B007B5E-E2E7-41B3-A62F-BBB5F886A0C2@iki.fi> <4FF16A6A.8070209@um.es> Message-ID: <00DEF5E6-A84F-4813-B22B-AC8DC14A8A58@iki.fi> On 2.7.2012, at 12.31, Angel L. Mateo wrote: >>>> My problem was that this timeout seems to be counted from the beginning of the LMTP connection, so when I have a lot of recipients in the same connection, last ones sometimes timedout. I solved it increasing this timeout with proxy_timeout option and reducing max number of lmtp recipients in postfix. >>> >>> Ah, interesting. These should help: >>> >>> http://hg.dovecot.org/dovecot-2.1/rev/27dccff46fe9 >>> http://hg.dovecot.org/dovecot-2.1/rev/8a97daa8aff6 >> >> Plus http://hg.dovecot.org/dovecot-2.1/rev/569588ff7ef0 although I'm not entirely sure if it's needed. The LMTP code is rather ugly and difficult to follow.. >> > Are these patches included in any released? (I have 2.1.5) I wrote those 3 hours ago, so no. From ewald.lists at fun.de Mon Jul 2 14:06:19 2012 From: ewald.lists at fun.de (Ewald Dieterich) Date: Mon, 02 Jul 2012 13:06:19 +0200 Subject: [Dovecot] UTF-8 mailbox name STATUS response In-Reply-To: <7A8C8BD1-3C7E-4E7B-BCCB-9A6CBD7C698C@iki.fi> References: <4FF164B8.7000107@fun.de> <7A8C8BD1-3C7E-4E7B-BCCB-9A6CBD7C698C@iki.fi> Message-ID: <4FF180AB.4040905@fun.de> On 07/02/12 11:25, Timo Sirainen wrote: > On 2.7.2012, at 12.07, Ewald Dieterich wrote: >> STATUS in Dovecot 2.1.7 returns the UTF-8 decoded folder name in a >> string literal: >> >> . CREATE "INBOX.Euro &IKw-" >> . OK Create completed. >> . LIST "" "INBOX.Euro &IKw-" >> * LIST (\HasNoChildren) "." "INBOX.Euro &IKw-" >> . STATUS "INBOX.Euro &IKw-" (MESSAGES) >> * STATUS {14} >> INBOX.Euro ? (MESSAGES 0) >> . OK Status completed. >> >> Is this intended? My Perl script based on Mail::IMAPTalk doesn't like >> it and I wonder if there are other clients that don't cope with it. > > Most likely that mailbox exists like that in the filesystem. It > shouldn't. Dovecot hasn't allowed creating those for several years > now. In the filesystem the mailbox is in mUTF-7: # ls mailboxes/ Euro &IKw-/ INBOX/ Trash/ > There's no good way to handle that, because even if Dovecot > translated it to mUTF-7 it couldn't access the mailbox because it > wouldn't exist as mUTF-7 in the filesystem.. To create the LIST response from my example above you do exactly this: convert the mailbox name to mUTF-7. . LIST "" "INBOX.Euro &IKw-" * LIST (\HasNoChildren) "." "INBOX.Euro &IKw-" In file imap/cmd-list.c: static int list_namespace_mailboxes(struct cmd_list_context *ctx) [...] if (imap_utf8_to_utf7(name, mutf7_name) < 0) i_panic("LIST: Mailbox name not UTF-8: %s", name); [...] imap_quote_append_string(str, str_c(mutf7_name), FALSE); So maybe you could do this for the STATUS response, too? Thanks for your help! From tss at iki.fi Mon Jul 2 14:58:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 2 Jul 2012 14:58:51 +0300 Subject: [Dovecot] UTF-8 mailbox name STATUS response In-Reply-To: <4FF180AB.4040905@fun.de> References: <4FF164B8.7000107@fun.de> <7A8C8BD1-3C7E-4E7B-BCCB-9A6CBD7C698C@iki.fi> <4FF180AB.4040905@fun.de> Message-ID: <63DA91C9-7946-4273-8AE3-6B450A0054A2@iki.fi> On 2.7.2012, at 14.06, Ewald Dieterich wrote: > >> . CREATE "INBOX.Euro &IKw-" > >> . OK Create completed. > >> . LIST "" "INBOX.Euro &IKw-" > >> * LIST (\HasNoChildren) "." "INBOX.Euro &IKw-" > >> . STATUS "INBOX.Euro &IKw-" (MESSAGES) > >> * STATUS {14} > >> INBOX.Euro ? (MESSAGES 0) > >> . OK Status completed. > >> > >> Is this intended? My Perl script based on Mail::IMAPTalk doesn't like > >> it and I wonder if there are other clients that don't cope with it. > > > > Most likely that mailbox exists like that in the filesystem. It > > shouldn't. Dovecot hasn't allowed creating those for several years > > now. > > In the filesystem the mailbox is in mUTF-7: Oh, sorry. I didn't even consider that it could have been broken for this long in v2.1.x without anyone noticing. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/a698fccd37c3 Time for v2.1.8 soon I guess :) From sam at robots.org.uk Mon Jul 2 15:34:19 2012 From: sam at robots.org.uk (Sam Morris) Date: Mon, 02 Jul 2012 13:34:19 +0100 Subject: [Dovecot] [PATCH] GSSAPI authorization and virtual users In-Reply-To: <1340787859.25551.47.camel@innu> References: <1330973136.70967.33.camel@leela.office.red-redemption.com> <1331057521.84875.2.camel@leela.office.red-redemption.com> <1340787859.25551.47.camel@innu> Message-ID: <1341232459.424390.0.camel@leela.office.red-redemption.com> On Wed, 2012-06-27 at 12:04 +0300, Timo Sirainen wrote: > On Tue, 2012-03-06 at 18:12 +0000, Sam Morris wrote: > > On Mon, 2012-03-05 at 20:52 +0200, Timo Sirainen wrote: > > > On 5.3.2012, at 20.45, Sam Morris wrote: > > > > > > > 3. The credentials lookup triggers an info log message saying that > > > > credentials for GSSAPI were requested, "but we have only (e.g.) > > > > MD5-CRYPT". The authplugin doesn't actually want the credential, > > > > but I think that the only way the authplugin can trigger a > > > > passdb lookup is by requesting it. > > > > > > I'll look at the rest more closely later, but this should be an easy fix: request "" instead of "GSSAPI". > > > > Thanks for pointing that out. Here's a newer version of the patch with > > that change. I also realised that the gss_buffer is not required in the > > code that runs once the passdb lookup is complete, so I removed the code > > that stashes it in struct gssapi_auth_request. > > I finally looked into this and did some changes. Does it still work? :) > http://hg.dovecot.org/dovecot-2.2/rev/183adc90781c Works fine, thanks! -- Sam Morris From voytek at sbt.net.au Mon Jul 2 16:23:40 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 2 Jul 2012 23:23:40 +1000 Subject: [Dovecot] + sub addressing setup Message-ID: I have Dovecot/Postfix/MySQL/PostfixAdmin, I'd like to setup + sub addressing (for +spam and possibly user-defined +folders) I have setup a user+spam folder path in PostfixAdmin as so; vvv+spam at tld path:tld/vvv at tld/.spam/ that requires me to put a 'duplicate' user with the + and +path for each user, the folks on PostfixAdmin suggested that Dovecot might have a better way of handling this, how do I accomplish this in Dovecot ? thanks for any suggestions or tips -- Voytek From voytek at sbt.net.au Mon Jul 2 16:28:07 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 2 Jul 2012 23:28:07 +1000 Subject: [Dovecot] + sub addressing setup In-Reply-To: References: Message-ID: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> On Mon, July 2, 2012 11:23 pm, Voytek Eymont wrote: > I have Dovecot/Postfix/MySQL/PostfixAdmin, sorry, forgot to include conf setup: doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.23.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) auth_cache_size = 1 k auth_mechanisms = plain login disable_plaintext_auth = no log_timestamp = "%Y-%m-%d %H:%M:%S " mail_location = maildir:~/Maildir mail_privileged_group = mail mbox_write_locks = fcntl namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } passdb { driver = pam } plugin { autocreate = Spam autosubscribe = Spam } protocols = imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } user = root } ssl = required ssl_cert = Hi, I use OpenBSD 5.1, roundcube 0.7.2 and dovecot 2.0.17 (684381041dc4+), mysql. I get the following error when i try to connect to imap : roundcube: IMAP Error: Login failed for testing at mydomain.net from 192.168.0.92. AUTHENTICATE PLAIN: Authentication failed. in /var/www/webmail/roundcubemail-0.7.2/program/include/rcube_imap.php on line 205 (POST /webmail/?_task=login&_action=login) dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=, method=CRAM-MD5, rip=127.0.0.1, lip=127.0.0.1, TLS Jul 2 15:19:13 mx dovecot: auth-worker: mysql(localhost): Connected to database mail Jul 2 15:19:15 mx dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Can you help me please ? Here my config files : *dovecot-sql.conf driver = mysql connect = host=localhost dbname=mail user=postfix password=postfix default_pass_scheme = PLAIN password_query = SELECT email as user, password FROM users WHERE email = '%u' user_query = SELECT id as uid, id as gid, home, concat('*:storage=', quota, 'M') AS quota_rule FROM users WHERE email = '%u' *dovecot.conf #i use this on OpenBSD 5.0 and dovecot 2.0.13, works, but it doesn't seem to work with this version :-( # 2.0.13: /etc/dovecot/dovecot.conf # OS: OpenBSD 5.0 i386 ffs auth_mechanisms = plain login digest-md5 cram-md5 apop auth_debug=yes base_dir = /var/dovecot/ first_valid_uid = 1000 mail_location = maildir:/var/mailserv/mail/%d/%n mmap_disable = yes passdb { args = /etc/dovecot-sql.conf driver = sql } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/local/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items; Deleted Messages autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir quota_rule = *:storage=5G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2 = storage=80%% /usr/local/bin/quota-warning.sh 80 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap sieve pop3 service auth { unix_listener /var/run/dovecot-auth-master { group = _dovecot mode = 0666 user = _dovecot } unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } } service imap-login { service_count = 0 #user = _dovecot vsz_limit = 64 M } service pop3-login { service_count = 0 #user = _dovecot vsz_limit = 64 M } ssl_cert =ssl_cipher_list = HIGH:MEDIUM:+TLSv1:!SSLv2:+SSLv3 ssl_key = userdb { args = /etc/dovecot-sql.conf driver = sql } #userdb { # driver = passwd #} protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota autocreate } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xv%08Xu } protocol lda { auth_socket_path = /var/run/dovecot-auth-master mail_plugins = $mail_plugins sieve postmaster_address = postmaster at mx.itdebug.net sendmail_path = /usr/sbin/sendmail } default_login_user = _dovenull default_internal_user = _dovecot ## ## ManageSieve specific settings ## # Service definitions service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service managesieve { #Max. number of ManageSieve processes (connections) #process_count = 1024 } # Service configuration protocol sieve { } Thank you very much. -- Wesley From micah at riseup.net Mon Jul 2 17:25:25 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 02 Jul 2012 10:25:25 -0400 Subject: [Dovecot] mailboxes missing after upgrade from 2.1.4 to 2.1.7 Message-ID: <87txxqxmsq.fsf@algae.riseup.net> Hi all, I mentioned this on the #dovecot irc channel, but I thought I would post here so I can provide more details. The basic problem is that when I upgraded from 2.1.4 to 2.1.7, some users no longer are able to see their folders in pine/alpine. The folders are actually there, and are subscribed (according to doveadm) and its possible to do operations on those folders, if you know the name (for example, opening the folder works fine if you know the name). When I downgrade to 2.1.4 again, the folders appear again, so its clearly something that changed between the two versions. I looked through the changelog, and found a few possibile culprits, I think pine/alpine use imapc, so that could be related: . imapc: Use imapc_list_prefix also for listing subscriptions . Fixed looking up parent mailbox name with different namespace vs. layout separators. . imapc: Don't list mailboxes that don't match list patterns. . layout=fs: Fixed listing mailboxes with prefix=INBOX/ . LIST with mailbox {} settings incorrectly set subscribed-flags. Here is my dovecot -n output: # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_default_realm = example.net auth_verbose = yes default_process_limit = 256 default_vsz_limit = 512 M dict { expire = mysql:/etc/dovecot/dovecot-dict-sql.conf quota = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no first_valid_gid = 8 first_valid_uid = 8 last_valid_gid = 8 last_valid_uid = 8 listen = * login_greeting = howdy, ready. mail_location = mdbox:~/mdbox:INDEX=/srv/dovecot_indexes/%d/%1n/%n mail_plugins = expire quota zlib stats maildir_very_dirty_syncs = yes namespace { inbox = yes location = prefix = separator = . } namespace { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { antispam_allow_append_to_spam = yes antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /usr/local/bin/train_spam antispam_pipe_program_args = -d;10.0.1.109 antispam_pipe_program_notspam_arg = ham antispam_pipe_program_spam_arg = spam antispam_pipe_tmpdir = /var/tmp antispam_signature = X-Spam-Flag antispam_signature_missing = error antispam_spam_pattern_ignorecase = SPAM antispam_trash_pattern = trash;Trash;Deleted * expire = Trash expire2 = Trash/* expire3 = Spam expire_dict = proxy::expire quota = dict:Your mail quota::proxy::quota quota_rule = *:bytes=100663296 quota_rule2 = Trash:bytes=+20%% quota_rule3 = Spam:bytes=+10%% quota_rule4 = INBOX.Trash:bytes=+20%% quota_rule5 = INBOX.Spam:bytes=+10%% quota_rule6 = INBOX.restored_from_backups:bytes=+500%% sieve = ~/.dovecot.sieve sieve_after = /var/lib/dovecot/sieve/after.sieve sieve_before = /var/lib/dovecot/sieve/default.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at example.net protocols = " imap pop3" service anvil { unix_listener anvil { group = dovecot mode = 0660 user = root } } service auth { unix_listener auth-userdb { group = mail mode = 0600 user = mail } } service dict { unix_listener dict { group = mail mode = 0600 user = mail } } service imap-login { process_min_avail = 10 service_count = 0 vsz_limit = 512 M } service imap-postlogin { executable = script-login /usr/local/sbin/postlogin_imap } service imap { executable = imap imap-postlogin process_limit = 2048 } service pop3-login { process_min_avail = 10 service_count = 0 vsz_limit = 512 M } service pop3-postlogin { executable = script-login /usr/local/sbin/postlogin_pop } service pop3 { executable = pop3 pop3-postlogin } service stats { fifo_listener stats-mail { mode = 0600 user = mail } } ssl_cert = Hi, though this is a bit of a side question, has anybody had an issue while running Outlook 2010 with Dovecot? The reason why I am asking is that I have setup a Dovecot 2.1.7 server on FreeBSD which works fantastically with Thunderbird but Outlook seems to be twice as slow in transferring information across?? # dovecot -n # 2.1.7: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 8.2-RELEASE amd64 auth_debug = yes auth_mechanisms = plain ntlm login auth_use_winbind = yes auth_username_format = %n auth_verbose = yes auth_winbind_helper_path = /usr/local/bin/ntlm_auth disable_plaintext_auth = no info_log_path = /var/log/dovecot-info.log log_path = /var/log/dovecot.log mail_gid = mail_user mail_home = /mail/AD_Mail/%Ld/%Ln mail_location = maildir:~/Maildir mail_uid = mail_user passdb { args = failure_show_msg=yes driver = pam } pop3_fast_size_lookups = yes pop3_lock_session = yes pop3_no_flag_updates = yes protocols = imap pop3 ssl = no userdb { driver = static } Since (like most corporate organizations out there) we solely run Outlook coupled to Exchange, this excersize was meant to be a way of getting rid of PST files. We don't run out own Exchange however, and don't have any control over it either. My workaround was to simply use the Outlook GUI space to transfer emails between Exchange and Dovecot running the IMAPv4 protocol. For whatever reason Outlook is being really garbage about dealing with stuff and since I don't know Outlook or MS products very well (being your typical average OpenSource guy) I was wondering if there were any tweaks that could be made within Outlook to speed it up or in Dovecot to work better with Outlook? I guess one could get sidetracked into the argument of mdbox vs. Maildir from my config however, Thunderbird is really fast and transfers large amounts of data really easily. Reaches round 130Mbps using nload performance grapher, while Outlook only manages ~50kbps but spikes at 2-3Mbps on occassion. Can anyone offer any guidence or assistance in this matter?? Actually wherever I run Dovecot, including my servers at home, it is fast and reliable. Yes I know MS is the polar opposite of anything worth using however, my company won't change and I'm stuck banging my head against the wall while trying to get MS to interface with ANYTHING..... Regards, Kaya From giles at coochey.net Mon Jul 2 17:37:21 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 15:37:21 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: <4FF1B221.9050609@coochey.net> On 02/07/2012 15:34, Kaya Saman wrote: > Hi, > > though this is a bit of a side question, has anybody had an issue > while running Outlook 2010 with Dovecot? > > Yes, as far as being an IMAP client Outlook 2010 has not been any good for me, slow, hangs, various non-intuitive issues... Thunderbird works a treat and I use it all the time, until someone sends me a meeting request from an Outlook client. -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From lists at svrinformatica.it Mon Jul 2 17:40:06 2012 From: lists at svrinformatica.it (Mailing List SVR) Date: Mon, 02 Jul 2012 16:40:06 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: <4FF1B2C6.1090404@svrinformatica.it> Il 02/07/2012 16:34, Kaya Saman ha scritto: > Hi, > > though this is a bit of a side question, has anybody had an issue > while running Outlook 2010 with Dovecot? > > The reason why I am asking is that I have setup a Dovecot 2.1.7 server > on FreeBSD which works fantastically with Thunderbird but Outlook > seems to be twice as slow in transferring information across?? > > > # dovecot -n > # 2.1.7: /usr/local/etc/dovecot/dovecot.conf > # OS: FreeBSD 8.2-RELEASE amd64 > auth_debug = yes > auth_mechanisms = plain ntlm login > auth_use_winbind = yes > auth_username_format = %n > auth_verbose = yes > auth_winbind_helper_path = /usr/local/bin/ntlm_auth > disable_plaintext_auth = no > info_log_path = /var/log/dovecot-info.log > log_path = /var/log/dovecot.log > mail_gid = mail_user > mail_home = /mail/AD_Mail/%Ld/%Ln > mail_location = maildir:~/Maildir > mail_uid = mail_user > passdb { > args = failure_show_msg=yes > driver = pam > } > pop3_fast_size_lookups = yes > pop3_lock_session = yes > pop3_no_flag_updates = yes > protocols = imap pop3 > ssl = no > userdb { > driver = static > } > > > > > Since (like most corporate organizations out there) we solely run > Outlook coupled to Exchange, this excersize was meant to be a way of > getting rid of PST files. We don't run out own Exchange however, and > don't have any control over it either. > > > My workaround was to simply use the Outlook GUI space to transfer > emails between Exchange and Dovecot running the IMAPv4 protocol. > > > For whatever reason Outlook is being really garbage about dealing with > stuff and since I don't know Outlook or MS products very well (being > your typical average OpenSource guy) I was wondering if there were any > tweaks that could be made within Outlook to speed it up or in Dovecot > to work better with Outlook? > > > I guess one could get sidetracked into the argument of mdbox vs. > Maildir from my config however, Thunderbird is really fast and > transfers large amounts of data really easily. Reaches round 130Mbps > using nload performance grapher, while Outlook only manages ~50kbps > but spikes at 2-3Mbps on occassion. Try to understand (from dovecot logs) if there are difference between outlook and thunderbird, for example outlook connect over ssl and thunderbird no ecc.. Nicola > > > Can anyone offer any guidence or assistance in this matter?? > > > Actually wherever I run Dovecot, including my servers at home, it is > fast and reliable. Yes I know MS is the polar opposite of anything > worth using however, my company won't change and I'm stuck banging my > head against the wall while trying to get MS to interface with > ANYTHING..... > > > Regards, > > > Kaya > From kayasaman at gmail.com Mon Jul 2 17:42:42 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Mon, 2 Jul 2012 15:42:42 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1B221.9050609@coochey.net> References: <4FF1B221.9050609@coochey.net> Message-ID: On Mon, Jul 2, 2012 at 3:37 PM, Giles Coochey wrote: > On 02/07/2012 15:34, Kaya Saman wrote: >> >> Hi, >> >> though this is a bit of a side question, has anybody had an issue >> while running Outlook 2010 with Dovecot? >> >> > Yes, as far as being an IMAP client Outlook 2010 has not been any good for > me, slow, hangs, various non-intuitive issues... > > Thunderbird works a treat and I use it all the time, until someone sends me > a meeting request from an Outlook client. > > -- > Regards, > > Giles Coochey, CCNA, CCNAS > NetSecSpec Ltd > +44 (0) 7983 877438 > http://www.coochey.net > http://www.netsecspec.co.uk > giles at coochey.net > > Thanks Giles, but unfortunately as stated before T-Bird won't connect to our remote Exchange server which doesn't have IMAP active for various reasons...... I prefer Alpine myself but try running that on Windows 7 (forced to at work :-( ). Regards, Kaya From kayasaman at gmail.com Mon Jul 2 17:51:34 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Mon, 2 Jul 2012 15:51:34 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1B2C6.1090404@svrinformatica.it> References: <4FF1B2C6.1090404@svrinformatica.it> Message-ID: On Mon, Jul 2, 2012 at 3:40 PM, Mailing List SVR wrote: > Il 02/07/2012 16:34, Kaya Saman ha scritto: > >> Hi, >> >> though this is a bit of a side question, has anybody had an issue >> while running Outlook 2010 with Dovecot? >> >> The reason why I am asking is that I have setup a Dovecot 2.1.7 server >> on FreeBSD which works fantastically with Thunderbird but Outlook >> seems to be twice as slow in transferring information across?? >> >> >> # dovecot -n >> # 2.1.7: /usr/local/etc/dovecot/dovecot.conf >> # OS: FreeBSD 8.2-RELEASE amd64 >> auth_debug = yes >> auth_mechanisms = plain ntlm login >> auth_use_winbind = yes >> auth_username_format = %n >> auth_verbose = yes >> auth_winbind_helper_path = /usr/local/bin/ntlm_auth >> disable_plaintext_auth = no >> info_log_path = /var/log/dovecot-info.log >> log_path = /var/log/dovecot.log >> mail_gid = mail_user >> mail_home = /mail/AD_Mail/%Ld/%Ln >> mail_location = maildir:~/Maildir >> mail_uid = mail_user >> passdb { >> args = failure_show_msg=yes >> driver = pam >> } >> pop3_fast_size_lookups = yes >> pop3_lock_session = yes >> pop3_no_flag_updates = yes >> protocols = imap pop3 >> ssl = no >> userdb { >> driver = static >> } >> >> >> >> >> Since (like most corporate organizations out there) we solely run >> Outlook coupled to Exchange, this excersize was meant to be a way of >> getting rid of PST files. We don't run out own Exchange however, and >> don't have any control over it either. >> >> >> My workaround was to simply use the Outlook GUI space to transfer >> emails between Exchange and Dovecot running the IMAPv4 protocol. >> >> >> For whatever reason Outlook is being really garbage about dealing with >> stuff and since I don't know Outlook or MS products very well (being >> your typical average OpenSource guy) I was wondering if there were any >> tweaks that could be made within Outlook to speed it up or in Dovecot >> to work better with Outlook? >> >> >> I guess one could get sidetracked into the argument of mdbox vs. >> Maildir from my config however, Thunderbird is really fast and >> transfers large amounts of data really easily. Reaches round 130Mbps >> using nload performance grapher, while Outlook only manages ~50kbps >> but spikes at 2-3Mbps on occassion. > > > Try to understand (from dovecot logs) if there are difference between > outlook and thunderbird, for example outlook connect over ssl and > thunderbird no ecc.. > > Nicola > > >> >> > > Hi Nicola, there is no specific difference apart from seeing many of these errors: Jun 26 15:10:11 imap(): Error: Index /mail/AD_Mail///Maildir/.Archive/dovecot.index: Lost log for seq=2 offset=77008 Jun 26 15:10:11 imap(): Warning: fscking index file /mail/AD_Mail///Maildir/.Archive/dovecot.index Jun 26 15:10:11 imap(): Error: Fixed index file /mail/AD_Mail///Maildir/.Archive/dovecot.index: log_file_seq 2 -> 3 Jun 26 15:10:26 auth: Error: Got NTLMSSP neg_flags=0xa2088207 Jun 26 15:10:26 auth: Error: Got user=[] domain=[] workstation=[WKS-41] len1=24 len2=278 Jun 26 15:10:26 auth: Error: Login for user []\[]@[WKS-41] failed due to [Reading winbind reply failed!] Jun 26 15:10:32 auth: Error: Got NTLMSSP neg_flags=0xa2088207 Regards, Kaya From robert at schetterer.org Mon Jul 2 18:00:07 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 02 Jul 2012 17:00:07 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: <4FF1B777.3050306@schetterer.org> Am 02.07.2012 16:34, schrieb Kaya Saman: > Hi, > > though this is a bit of a side question, has anybody had an issue > while running Outlook 2010 with Dovecot? > > The reason why I am asking is that I have setup a Dovecot 2.1.7 server > on FreeBSD which works fantastically with Thunderbird but Outlook > seems to be twice as slow in transferring information across?? Hi, must be your setup no Problems here with Outlook 2010, sorry no time recent for analyse your posted config -- Best Regards MfG Robert Schetterer From giles at coochey.net Mon Jul 2 18:02:29 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 16:02:29 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> Message-ID: <4FF1B805.10603@coochey.net> On 02/07/2012 15:51, Kaya Saman wrote: > > Hi Nicola, > > there is no specific difference apart from seeing many of these errors: > > Jun 26 15:10:11 imap(): Error: Index > /mail/AD_Mail///Maildir/.Archive/dovecot.index: Lost log for > seq=2 offset=77008 > Jun 26 15:10:11 imap(): Warning: fscking index file > /mail/AD_Mail///Maildir/.Archive/dovecot.index > Jun 26 15:10:11 imap(): Error: Fixed index file > /mail/AD_Mail///Maildir/.Archive/dovecot.index: log_file_seq 2 > -> 3 > Jun 26 15:10:26 auth: Error: Got NTLMSSP neg_flags=0xa2088207 > Jun 26 15:10:26 auth: Error: Got user=[] domain=[] > workstation=[WKS-41] len1=24 len2=278 > Jun 26 15:10:26 auth: Error: Login for user []\[]@[WKS-41] > failed due to [Reading winbind reply failed!] > Jun 26 15:10:32 auth: Error: Got NTLMSSP neg_flags=0xa2088207 > > > From the Dovecot wiki about ntlm_auth: Dovecot currently does blocking lookups, so if ntlm_auth is slow on responding (e.g. network problems), Dovecot blocks all other authentication requests until it's finished. Just wondering if Outlook parallelises it's sessions and ntlm_auth is blocking because of the timing between the requests. Have you tried a test user with local authentication? -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From giles at coochey.net Mon Jul 2 18:06:02 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 16:06:02 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1B805.10603@coochey.net> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> Message-ID: <4FF1B8DA.3060200@coochey.net> On 02/07/2012 16:02, Giles Coochey wrote: > On 02/07/2012 15:51, Kaya Saman wrote: >> >> Hi Nicola, >> >> there is no specific difference apart from seeing many of these errors: >> >> Jun 26 15:10:11 imap(): Error: Index >> /mail/AD_Mail///Maildir/.Archive/dovecot.index: Lost log for >> seq=2 offset=77008 >> Jun 26 15:10:11 imap(): Warning: fscking index file >> /mail/AD_Mail///Maildir/.Archive/dovecot.index >> Jun 26 15:10:11 imap(): Error: Fixed index file >> /mail/AD_Mail///Maildir/.Archive/dovecot.index: log_file_seq 2 >> -> 3 >> Jun 26 15:10:26 auth: Error: Got NTLMSSP neg_flags=0xa2088207 >> Jun 26 15:10:26 auth: Error: Got user=[] domain=[] >> workstation=[WKS-41] len1=24 len2=278 >> Jun 26 15:10:26 auth: Error: Login for user []\[]@[WKS-41] >> failed due to [Reading winbind reply failed!] >> Jun 26 15:10:32 auth: Error: Got NTLMSSP neg_flags=0xa2088207 >> >> >> > From the Dovecot wiki about ntlm_auth: > > Dovecot currently does blocking lookups, so if ntlm_auth is slow on > responding (e.g. network problems), Dovecot blocks all other > authentication requests until it's finished. > > Just wondering if Outlook parallelises it's sessions and ntlm_auth is > blocking because of the timing between the requests. > > Have you tried a test user with local authentication? > Or perhaps try auth_cache_size = 1024 To cache authentications. -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From kayasaman at gmail.com Mon Jul 2 18:21:22 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Mon, 2 Jul 2012 16:21:22 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1B8DA.3060200@coochey.net> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> Message-ID: On Mon, Jul 2, 2012 at 4:06 PM, Giles Coochey wrote: > On 02/07/2012 16:02, Giles Coochey wrote: >> >> On 02/07/2012 15:51, Kaya Saman wrote: >>> >>> >>> Hi Nicola, >>> >>> there is no specific difference apart from seeing many of these errors: >>> >>> Jun 26 15:10:11 imap(): Error: Index >>> /mail/AD_Mail///Maildir/.Archive/dovecot.index: Lost log for >>> seq=2 offset=77008 >>> Jun 26 15:10:11 imap(): Warning: fscking index file >>> /mail/AD_Mail///Maildir/.Archive/dovecot.index >>> Jun 26 15:10:11 imap(): Error: Fixed index file >>> /mail/AD_Mail///Maildir/.Archive/dovecot.index: log_file_seq 2 >>> -> 3 >>> Jun 26 15:10:26 auth: Error: Got NTLMSSP neg_flags=0xa2088207 >>> Jun 26 15:10:26 auth: Error: Got user=[] domain=[] >>> workstation=[WKS-41] len1=24 len2=278 >>> Jun 26 15:10:26 auth: Error: Login for user []\[]@[WKS-41] >>> failed due to [Reading winbind reply failed!] >>> Jun 26 15:10:32 auth: Error: Got NTLMSSP neg_flags=0xa2088207 >>> >>> >>> >> From the Dovecot wiki about ntlm_auth: >> >> Dovecot currently does blocking lookups, so if ntlm_auth is slow on >> responding (e.g. network problems), Dovecot blocks all other authentication >> requests until it's finished. >> >> Just wondering if Outlook parallelises it's sessions and ntlm_auth is >> blocking because of the timing between the requests. >> >> Have you tried a test user with local authentication? >> > Or perhaps try > > auth_cache_size = 1024 > > To cache authentications. > > > -- > Regards, > > Giles Coochey, CCNA, CCNAS > NetSecSpec Ltd > +44 (0) 7983 877438 > http://www.coochey.net > http://www.netsecspec.co.uk > giles at coochey.net > > Thanks Giles :-) I think that has made things a bit better...... In terms of the cache, is that measured in seconds?? Regards, Kaya From giles at coochey.net Mon Jul 2 18:22:49 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 16:22:49 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> Message-ID: <4FF1BCC9.9000301@coochey.net> On 02/07/2012 16:21, Kaya Saman wrote: > >> Or perhaps try >> >> auth_cache_size = 1024 >> >> To cache authentications. >> >> >> -- >> Regards, >> >> Giles Coochey, CCNA, CCNAS >> NetSecSpec Ltd >> +44 (0) 7983 877438 >> http://www.coochey.net >> http://www.netsecspec.co.uk >> giles at coochey.net >> >> > Thanks Giles :-) > > I think that has made things a bit better...... > > In terms of the cache, is that measured in seconds?? > > > Regards, > > Kaya The size is in KB. I'm afraid cache-timeout and the inner workings would be something only Timo or the Source Code know :-) -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From giles at coochey.net Mon Jul 2 18:31:12 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 16:31:12 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1BCC9.9000301@coochey.net> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> Message-ID: <4FF1BEC0.4020007@coochey.net> On 02/07/2012 16:22, Giles Coochey wrote: > The size is in KB. I'm afraid cache-timeout and the inner workings > would be something only Timo or the Source Code know :-) > http://wiki2.dovecot.org/Authentication/Caching the TTL setting is in seconds - perhaps what you are looking for? -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.netsecspec.co.uk giles.coochey at netsecspec.co.uk -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 18:39:14 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 17:39:14 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: > though this is a bit of a side question, has anybody had an issue while > running Outlook 2010 with Dovecot? No because i don't use that shit and enforce anyone not to do this. Outlook is terrible and even worse with imap. It is not Dovecot fault and have nothing to Dovecot. Just replace Outlook with real MUA that actually works. If someone wants to continue using that shit then it is not your problem but his/her problem, and all responsibility of doing this should go to the user. The only tweak is to install real mail client. period. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 18:40:43 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 17:40:43 +0200 (CEST) Subject: [Dovecot] RAID1+md concat+XFS as mailstorage In-Reply-To: <4FEFF98F.4020409@hardwarefreak.com> References: <213B51F00051AE48A9F0E11288017717B8401D@Delta.sc.local><4FEC4ACD.20104@wildgooses.com> <4FEE8D6E.3030703@hardwarefreak.com> <213B51F00051AE48A9F0E11288017717B84022@Delta.sc.local> <4FEFF98F.4020409@hardwarefreak.com> Message-ID: > > No, I'm saying you are trolling. A concat of RAID1 pairs has > reliability identical to RAID10. not a concat but separate filesystem. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 18:41:32 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 17:41:32 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B221.9050609@coochey.net> Message-ID: > I prefer Alpine myself but try running that on Windows 7 (forced to at > work :-( ). never needed but at least under windows XP it works fine. From kayasaman at gmail.com Mon Jul 2 18:43:18 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Mon, 2 Jul 2012 16:43:18 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1BEC0.4020007@coochey.net> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> Message-ID: On Mon, Jul 2, 2012 at 4:31 PM, Giles Coochey wrote: > On 02/07/2012 16:22, Giles Coochey wrote: >> >> The size is in KB. I'm afraid cache-timeout and the inner workings >> would be something only Timo or the Source Code know :-) >> > http://wiki2.dovecot.org/Authentication/Caching > > the TTL setting is in seconds - perhaps what you are looking for? > > > > -- > Regards, > > Giles Coochey, CCNA, CCNAS > NetSecSpec Ltd > +44 (0) 7983 877438 > http://www.netsecspec.co.uk > giles.coochey at netsecspec.co.uk > > > -- > Regards, > > Giles Coochey, CCNA, CCNAS > NetSecSpec Ltd > +44 (0) 7983 877438 > http://www.coochey.net > http://www.netsecspec.co.uk > giles at coochey.net > > Thanks for that! I added this: auth_cache_size = 4096 auth_cache_ttl = 14400 to my config and checked with dovecot -n I actually managed to knock 13 seconds of the original transfer time! :-) Good but not good enough especially when some of our users have round 20GB of PST file :-( Regards, Kaya From odhiambo at gmail.com Mon Jul 2 18:44:43 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Mon, 2 Jul 2012 18:44:43 +0300 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: On Mon, Jul 2, 2012 at 6:39 PM, Wojciech Puchar < wojtek at wojtek.tensor.gdynia.pl> wrote: > though this is a bit of a side question, has anybody had an issue while >> running Outlook 2010 with Dovecot? >> > No because i don't use that shit and enforce anyone not to do this. > > Outlook is terrible and even worse with imap. It is not Dovecot fault and > have nothing to Dovecot. > > Just replace Outlook with real MUA that actually works. > > If someone wants to continue using that shit then it is not your problem > but his/her problem, and all responsibility of doing this should go to the > user. > > The only tweak is to install real mail client. period. > Wojciech, I believe you do recognize that this may be something that requires policy changes to take effect. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. From jerry at seibercom.net Mon Jul 2 18:52:34 2012 From: jerry at seibercom.net (Jerry) Date: Mon, 2 Jul 2012 11:52:34 -0400 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1B777.3050306@schetterer.org> References: <4FF1B777.3050306@schetterer.org> Message-ID: <20120702115234.1587d728@scorpio> On Mon, 02 Jul 2012 17:00:07 +0200 Robert Schetterer articulated: > Am 02.07.2012 16:34, schrieb Kaya Saman: > > Hi, > > > > though this is a bit of a side question, has anybody had an issue > > while running Outlook 2010 with Dovecot? > > > > The reason why I am asking is that I have setup a Dovecot 2.1.7 > > server on FreeBSD which works fantastically with Thunderbird but > > Outlook seems to be twice as slow in transferring information > > across?? > > Hi, must be your setup no Problems here with > Outlook 2010, sorry no time recent for analyse your posted config I don't have any problems with it either. sounds like it could be a networking problem. I have also heard on the Postfix list about some AV programs causing problems. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From CMarcus at Media-Brokers.com Mon Jul 2 18:54:07 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 02 Jul 2012 11:54:07 -0400 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: <4FF1C41F.4060200@Media-Brokers.com> On 2012-07-02 11:39 AM, Wojciech Puchar wrote: >> though this is a bit of a side question, has anybody had an issue >> while running Outlook 2010 with Dovecot? > No because i don't use that shit and enforce anyone not to do this. > > Outlook is terrible and even worse with imap. It is not Dovecot fault > and have nothing to Dovecot. > > Just replace Outlook with real MUA that actually works. Please don't be an ass - if you can't help with the actual problem, just keep comments like this to yourself. Many companies require Outlook, and the fact is, as an EXCHANGE client, Outlook works extremely well. I agree that as a standalone IMAP client it isn't the best, but 2010 is much better than 2003 and earlier versions... -- Best regards, Charles From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 18:54:28 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 17:54:28 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: > Wojciech, > > I believe you do recognize that this may be something that requires policy changes to take effect. Of course i do! If you are not the one deciding with policy then state clearly that this shit simply doesn't work, so if the policy is to use it, then the same policy should state "don't expect e-mail to work". No policy can override truth and facts. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 18:55:41 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 17:55:41 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <20120702115234.1587d728@scorpio> References: <4FF1B777.3050306@schetterer.org> <20120702115234.1587d728@scorpio> Message-ID: >> Hi, must be your setup no Problems here with >> Outlook 2010, sorry no time recent for analyse your posted config > > I don't have any problems with it either. sounds like it could be a > networking problem. I have also heard on the Postfix list about some AV > programs causing problems. If you don't have problems with outlook then you have very little mail per user. Try this shit with 20GB mailbox. It is really funny i promise :) From giles at coochey.net Mon Jul 2 18:58:20 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 16:58:20 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: <4FF1C51C.8060509@coochey.net> On 02/07/2012 16:54, Wojciech Puchar wrote: >> Wojciech, >> >> I believe you do recognize that this may be something that requires >> policy changes to take effect. > Of course i do! > > If you are not the one deciding with policy then state clearly that > this shit simply doesn't work, so if the policy is to use it, then the > same policy should state "don't expect e-mail to work". > > No policy can override truth and facts. > I'm not going to comment other than say that this sub-thread probably needs to continue on: alt.flames.anti-microsoft.linux.jihad They love these types of thread there :-D -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From bdh at machinehum.com Mon Jul 2 18:59:12 2012 From: bdh at machinehum.com (Brian Hayden) Date: Mon, 2 Jul 2012 10:59:12 -0500 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: On Jul 2, 2012, at 10:54 AM, Wojciech Puchar wrote: >> Wojciech, >> I believe you do recognize that this may be something that requires policy changes to take effect. > Of course i do! > > If you are not the one deciding with policy then state clearly that this shit simply doesn't work, so if the policy is to use it, then the same policy should state "don't expect e-mail to work". Speaking of truth and facts, you've had a lot of advice here lately for someone who clearly has never worked on anything but toy projects with users that you're free to bully into submission. If you don't have something useful to contribute, why not just keep it to yourself? -Brian From odhiambo at gmail.com Mon Jul 2 19:00:32 2012 From: odhiambo at gmail.com (Odhiambo Washington) Date: Mon, 2 Jul 2012 19:00:32 +0300 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B777.3050306@schetterer.org> <20120702115234.1587d728@scorpio> Message-ID: On Mon, Jul 2, 2012 at 6:55 PM, Wojciech Puchar < wojtek at wojtek.tensor.gdynia.pl> wrote: > Hi, must be your setup no Problems here with >>> Outlook 2010, sorry no time recent for analyse your posted config >>> >> >> I don't have any problems with it either. sounds like it could be a >> networking problem. I have also heard on the Postfix list about some AV >> programs causing problems. >> > If you don't have problems with outlook then you have very little mail per > user. > > Try this shit with 20GB mailbox. It is really funny i promise :) > It's not funny at all, using certain references as you like doing. I don't remember when I last heard such words on this list. Please, if you cannot resist the temptation, s#s.*#crap#g. It keeps this list cleaner. -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254733744121/+254722743223 _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ I can't hear you -- I'm using the scrambler. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 19:06:13 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 18:06:13 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1C51C.8060509@coochey.net> References: <4FF1C51C.8060509@coochey.net> Message-ID: >> No policy can override truth and facts. >> > > I'm not going to comment other than say that this sub-thread probably needs > to continue on: > > alt.flames.anti-microsoft.linux.jihad no. no jihad. No linux actually (i don't use linux). That's fact. If someone want to use outlook then fine, but should not expect anyone else than microsoft to fix it's problems and speed. It is nothing to do with dovecot. All my experience with that shit now is remote. I quite often handle complaints from users that a) "my mail wasn't delivered to somebody." After checking logs i found that recipent's server received mail with success. But mail disappeared then. Sometimes reappeared after a week, sometimes many times, or never ;) The reason was microsoft exchange on recipient side. Sometimes - badly configured antispam. b) reverse - "i don't receive mail from someone". Because someone's MS exchange server didn't even try to send it. c) i'm getting some strange attachment that i cannot open. (winmail.dat) my answer: Ask sender to send attachments according to standards From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 19:06:48 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 18:06:48 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: > Speaking of truth and facts, you've had a lot of advice here lately for someone who clearly has never worked on anything but toy projects with users that you're free to bully into submission. If you don't have something useful to contribute, why not just keep it to yourself? If you show me outlook actually working on large projects then i will stop. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 19:07:38 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 18:07:38 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B777.3050306@schetterer.org> <20120702115234.1587d728@scorpio> Message-ID: > > It's not funny at all, using certain references as you like doing. I don't remember when I last heard such words on this list. What do you fear? From bdh at machinehum.com Mon Jul 2 19:09:13 2012 From: bdh at machinehum.com (Brian Hayden) Date: Mon, 2 Jul 2012 11:09:13 -0500 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: Message-ID: <2C454D6E-0B8C-435C-8BB7-C2891555F0DB@machinehum.com> On Jul 2, 2012, at 11:06 AM, Wojciech Puchar wrote: >> Speaking of truth and facts, you've had a lot of advice here lately for someone who clearly has never worked on anything but toy projects with users that you're free to bully into submission. If you don't have something useful to contribute, why not just keep it to yourself? > > If you show me outlook actually working on large projects then i will stop. You're not necessarily wrong about Outlook vis-a-vis IMAP. You're very wrong about how much power an email admin has in a real organization. Please take the non-constructive flaming and cursing somewhere else, as others have suggested. -Brian From kayasaman at gmail.com Mon Jul 2 19:12:16 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Mon, 2 Jul 2012 17:12:16 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1BEC0.4020007@coochey.net> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> Message-ID: On Mon, Jul 2, 2012 at 4:31 PM, Giles Coochey wrote: > On 02/07/2012 16:22, Giles Coochey wrote: >> >> The size is in KB. I'm afraid cache-timeout and the inner workings >> would be something only Timo or the Source Code know :-) >> > http://wiki2.dovecot.org/Authentication/Caching > > the TTL setting is in seconds - perhaps what you are looking for? > > > > -- > Regards, > > Giles Coochey, CCNA, CCNAS > NetSecSpec Ltd > +44 (0) 7983 877438 > http://www.netsecspec.co.uk > giles.coochey at netsecspec.co.uk > > > -- > Regards, > > Giles Coochey, CCNA, CCNAS > NetSecSpec Ltd > +44 (0) 7983 877438 > http://www.coochey.net > http://www.netsecspec.co.uk > giles at coochey.net > > Giles, what's really weird is that if I keep increasing the Cache TTL and Cache size, the speed of transfer starts dropping. Perhaps I haven't hit the sweet spot yet however, running: auth_cache_size = 160000 B auth_cache_ttl = 8 hours I am actually knocking off 4 seconds from half size values. To be honest I just whish I could understand what is going on in order to get the transfer sub-3 minutes! Regards, Kaya From gedalya at gedalya.net Mon Jul 2 19:50:00 2012 From: gedalya at gedalya.net (Gedalya) Date: Mon, 02 Jul 2012 12:50:00 -0400 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1C51C.8060509@coochey.net> Message-ID: <4FF1D138.6050801@gedalya.net> On 07/02/2012 12:06 PM, Wojciech Puchar wrote: >>> No policy can override truth and facts. >>> >> >> I'm not going to comment other than say that this sub-thread probably >> needs to continue on: >> >> alt.flames.anti-microsoft.linux.jihad > > no. no jihad. No linux actually (i don't use linux). > > That's fact. > If someone want to use outlook then fine, but should not expect anyone > else than microsoft to fix it's problems and speed. > > It is nothing to do with dovecot. > > All my experience with that shit now is remote. > > I quite often handle complaints from users that > > a) "my mail wasn't delivered to somebody." > > After checking logs i found that recipent's server received mail with > success. But mail disappeared then. > > Sometimes reappeared after a week, sometimes many times, or never ;) > > > The reason was microsoft exchange on recipient side. Sometimes - badly > configured antispam. > > b) reverse - "i don't receive mail from someone". > > Because someone's MS exchange server didn't even try to send it. > > c) i'm getting some strange attachment that i cannot open. (winmail.dat) > > my answer: Ask sender to send attachments according to standards > > Equivalents of all these can be presented the other way around against badly configured free software solutions. I'd rather say that the real problem is that Microsoft programs require the same level of knowledge and expertise that, ehhhm, computer systems require. They are just not presented that way, and create a culture of "oh sure I can do this, so easy". Exchange *can* be configured to actually deliver mail, it just attracts all the wrong admins. And capable admins will just naturally opt for other solutions. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 19:55:06 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 18:55:06 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <2C454D6E-0B8C-435C-8BB7-C2891555F0DB@machinehum.com> References: <2C454D6E-0B8C-435C-8BB7-C2891555F0DB@machinehum.com> Message-ID: > You're not necessarily wrong about Outlook vis-a-vis IMAP. You're very wrong about how much power an email admin has in a real organization. Please take the non-constructive flaming and cursing somewhere else, as others have suggested. Still you can't improve trash program by better IMAP server. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 2 20:10:29 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 2 Jul 2012 19:10:29 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1D138.6050801@gedalya.net> References: <4FF1C51C.8060509@coochey.net> <4FF1D138.6050801@gedalya.net> Message-ID: > Equivalents of all these can be presented the other way around against badly > configured free software solutions. Fortunately not microsoft sets open standard and have to conform to them or there will be microsoft only mail. And more fortunately not these admins. OK response from exchange server getting properly my e-mail is enough proof that email was delivered to it. No attempts (including tcpdump trace) of sending e-mail from exchange serverr to me is too a proof of email not being delivered by it. Yet fortunately it is really easy to fight it - EXPLAIN people. Most people just don't know what they are doing wrong and not willingly want to make things more complicated. Simply explaining them is from my experience enough in 99% of cases. YES really it works but most people here never tried it! As for latter - maybe it is possible to make exchange actually work, but statistics shows that it is not possible, or incredibly hard or people that are exchange administrators have no knowledge. For any of my clients asking if exchange or outlook would be good solution i just recommend visiting any company that actually use it and ask average employee (not boss). This always work. The other false statement is that such solution are designed for large scale businesses. The larger case the less chance it may work at all. Seems i am the only one here that do not fear the truth. I don't risk being fired :) as i work for those that (most often) already got at least one of that "enterprise" solutions and wanted something that actually work. And this is what i recommend to all of you. I recommend ending this topic altogether, as it is not dovecot related at all - unless anyone will find a problem with Dovecot implementation of IMAP that is clearly not confirming to standards, and (possibly) affect microsoft outlook. Highly unlikely but possible. Dovecot is high performance IMAP server, but will not fix bad client software. From robert at schetterer.org Mon Jul 2 20:21:25 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 02 Jul 2012 19:21:25 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> Message-ID: <4FF1D895.7000802@schetterer.org> Am 02.07.2012 17:43, schrieb Kaya Saman: > Good but not good enough especially when some of our users have round > 20GB of PST file :-( please describe where is the relation between a pst file and imap pst files are local after all having 20 GB PST File is a user Problem ever, tell them to split up by year etc beyond sizes under 2 GB for each folder its no problem to work wich many pst files -- Best Regards MfG Robert Schetterer From simon.buongiorno at gmail.com Mon Jul 2 20:28:10 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Mon, 2 Jul 2012 13:28:10 -0400 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1D895.7000802@schetterer.org> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <4FF1D895.7000802@schetterer.org> Message-ID: On 2 July 2012 13:21, Robert Schetterer wrote: > Am 02.07.2012 17:43, schrieb Kaya Saman: >> Good but not good enough especially when some of our users have round >> 20GB of PST file :-( > > please describe where is the relation between a pst file and imap > pst files are local > > after all having 20 GB PST File is a user Problem ever, tell them to > split up by year etc beyond sizes under 2 GB for each folder its no > problem to work wich many pst files And to add to Robert's excellent comments, perhaps the best policy change (since you're so keen on changing policy) would be educate your users to use email clients for email and not document storage/management systems. It's incredibly hard to get 20GB PSTs if they are stripping attachments. Simon From robert at schetterer.org Mon Jul 2 20:55:37 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 02 Jul 2012 19:55:37 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1C51C.8060509@coochey.net> <4FF1D138.6050801@gedalya.net> Message-ID: <4FF1E099.5010908@schetterer.org> Am 02.07.2012 19:10, schrieb Wojciech Puchar: >> Equivalents of all these can be presented the other way around against >> badly configured free software solutions. > > Fortunately not microsoft sets open standard and have to conform to them > or there will be microsoft only mail. And more fortunately not these > admins. > > OK response from exchange server getting properly my e-mail is enough > proof that email was delivered to it. > No attempts (including tcpdump trace) of sending e-mail from exchange > serverr to me is too a proof of email not being delivered by it. > > Yet fortunately it is really easy to fight it - EXPLAIN people. > > Most people just don't know what they are doing wrong and not willingly > want to make things more complicated. Simply explaining them is from my > experience enough in 99% of cases. YES really it works but most people > here never tried it! > > As for latter - maybe it is possible to make exchange actually work, but > statistics shows that it is not possible, or incredibly hard or people > that are exchange administrators have no knowledge. > > For any of my clients asking if exchange or outlook would be good > solution i just recommend visiting any company that actually use it and > ask average employee (not boss). This always work. > > The other false statement is that such solution are designed for large > scale businesses. The larger case the less chance it may work at all. > > Seems i am the only one here that do not fear the truth. I don't risk > being fired :) as i work for those that (most often) already got at > least one of that "enterprise" solutions and wanted something that > actually work. > > And this is what i recommend to all of you. > > > I recommend ending this topic altogether, as it is not dovecot related > at all - unless anyone will find a problem with Dovecot implementation > of IMAP that is clearly not confirming to standards, and (possibly) > affect microsoft outlook. > > Highly unlikely but possible. > > Dovecot is high performance IMAP server, but will not fix bad client > software. Hi , as i said million times before dont compare the incomparable Outlook is the client of Exchange , it can do smtp,imap,pop3 too its sold as a solution, with os , server, auth system ,client , support people etc There is no reason for M$ to make Qutlook fit in a perfect imap client cause this would goal in less reason for exchange Nevertheless if you have good admins and money to buy licences and good support, outlook and exchange are good company solutions for intranet mail and groupware ( like calendering etc ) dont flame by m$ is earning money, help making free solutions to get better specially in groupware functions with cross os clients like thunderbird etc dovecot is an imap server it has allready nearly all features comparable to the "exchange imap part", like folder acl etc nothing more is the job of dovecot customers want to do calendering with sharing ,adressbooks with sharings etc all with one client ( for sure mostly they are Outlook junkies ) but this isnt the real problem The real problem is that there arent full working and comparable groupware funktions in one client yet, also on the server side there arent complete free solutions cal/carddav is on the way, as well as syncml and other stuff so the goal is near to have all stuff needed to get comparable and much better meanwhile help dovecot users in fix their tec problems not in flame the shouldnt use a client whatever , most of the times this isnt what server admin can press sombody too Outlook works fine with dovecot if your setup is right dont think thunderbird is a bugless imap client At the end know your enemy, dont talk about outlook if you never worked around it, dont expect things from stuff which it was never made for solve -- Best Regards MfG Robert Schetterer From user+dovecot at localhost.localdomain.org Mon Jul 2 21:00:03 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 02 Jul 2012 20:00:03 +0200 Subject: [Dovecot] + sub addressing setup In-Reply-To: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> References: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> Message-ID: <4FF1E1A3.3010202@localhost.localdomain.org> See this this thread: http://dovecot.org/list/dovecot/2012-June/066374.html Regards, Pascal -- The trapper recommends today: c01dcafe.1218419 at localdomain.org From user+dovecot at localhost.localdomain.org Mon Jul 2 21:15:01 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 02 Jul 2012 20:15:01 +0200 Subject: [Dovecot] + sub addressing setup In-Reply-To: <4FF1E1A3.3010202@localhost.localdomain.org> References: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> <4FF1E1A3.3010202@localhost.localdomain.org> Message-ID: <4FF1E525.6020606@localhost.localdomain.org> On 07/02/2012 08:00 PM Pascal Volk wrote: > See this this thread: http://dovecot.org/list/dovecot/2012-June/066374.html > Oh, forget that tread. :) I hope you are using Dovecot's lmtp, then set lmtp_save_to_detail_mailbox = yes, in your 20-lmtp.conf. Otherwise pass use the -m option with dovecot-lda (http://wiki2.dovecot.org/Tools/DovecotLDA) Regards, Pascal -- The trapper recommends today: deadbeef.1218420 at localdomain.org From dovecot at list-post.mks-mail.de Mon Jul 2 22:49:11 2012 From: dovecot at list-post.mks-mail.de (=?UTF-8?B?TWFya3VzIFNjaMO2bmhhYmVy?=) Date: Mon, 02 Jul 2012 21:49:11 +0200 Subject: [Dovecot] Dovecot 2.1 stable packages for Debian? In-Reply-To: References: Message-ID: <4FF1FB37.50803@list-post.mks-mail.de> 11.06.2012 13:23, Krzysztof Trybowski: > it is strange, but Dovecot 2.x still didn't make it into Debian (not > even backports). It seems that 2.1.7 has landed in squeeze-backports. -- Regards mks From jerry at seibercom.net Mon Jul 2 23:14:29 2012 From: jerry at seibercom.net (Jerry) Date: Mon, 2 Jul 2012 16:14:29 -0400 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1C41F.4060200@Media-Brokers.com> References: <4FF1C41F.4060200@Media-Brokers.com> Message-ID: <20120702161429.5cd29802@scorpio> On Mon, 02 Jul 2012 11:54:07 -0400 Charles Marcus articulated: > On 2012-07-02 11:39 AM, Wojciech Puchar > wrote: > >> though this is a bit of a side question, has anybody had an issue > >> while running Outlook 2010 with Dovecot? > > > No because i don't use that shit and enforce anyone not to do this. > > > > Outlook is terrible and even worse with imap. It is not Dovecot > > fault and have nothing to Dovecot. > > > > Just replace Outlook with real MUA that actually works. > > Please don't be an ass - if you can't help with the actual problem, > just keep comments like this to yourself. > > Many companies require Outlook, and the fact is, as an EXCHANGE > client, Outlook works extremely well. I agree that as a standalone > IMAP client it isn't the best, but 2010 is much better than 2003 and > earlier versions... Wojciech has been Spamming up several forums lately. I finally set up a kill filter to eliminate him on those forums. I see that I am going to have to make one for the Dovecot mailing list also. Perhaps I should simple write a rule to have Postfix reject his mail. He NEVER has anything constructive to say and has been labeled a TROLL numerous times. His language is extremely vulgar and hateful. He likes to imply that he is knowledgeable yet fails to display any of that alleged knowledge. His abilities do not seem to fit within the pandect of modern system administration. Following his postings on other forums has led me to discover that he is simply a Microsoft antagonist with none of the required prerequisites to be one. I am sure his unnamed (mythical) company is proud of him. -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From giles at coochey.net Mon Jul 2 23:25:17 2012 From: giles at coochey.net (Giles Coochey) Date: Mon, 02 Jul 2012 21:25:17 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> Message-ID: <4FF203AD.2070502@coochey.net> On 02/07/2012 17:12, Kaya Saman wrote: > On Mon, Jul 2, 2012 at 4:31 PM, Giles Coochey wrote: >> On 02/07/2012 16:22, Giles Coochey wrote: >>> The size is in KB. I'm afraid cache-timeout and the inner workings >>> would be something only Timo or the Source Code know :-) >>> >> http://wiki2.dovecot.org/Authentication/Caching >> >> the TTL setting is in seconds - perhaps what you are looking for? >> >> >> >> -- >> Regards, >> >> Giles Coochey, CCNA, CCNAS >> NetSecSpec Ltd >> +44 (0) 7983 877438 >> http://www.netsecspec.co.uk >> giles.coochey at netsecspec.co.uk >> >> >> -- >> Regards, >> >> Giles Coochey, CCNA, CCNAS >> NetSecSpec Ltd >> +44 (0) 7983 877438 >> http://www.coochey.net >> http://www.netsecspec.co.uk >> giles at coochey.net >> >> > Giles, > > what's really weird is that if I keep increasing the Cache TTL and > Cache size, the speed of transfer starts dropping. > > Perhaps I haven't hit the sweet spot yet however, > > > running: > > auth_cache_size = 160000 B > auth_cache_ttl = 8 hours > > > I am actually knocking off 4 seconds from half size values. > > > To be honest I just whish I could understand what is going on in order > to get the transfer sub-3 minutes! > > > Regards, > > Kaya I would have thought that just enabling a simple cache would have given you a little performance increase, but unless you have a lot of users tweaking the values ought not to give you much more of a performance gain. In any case, the bottleneck appears to be your authentication mechanism, for which you're using a samba tool (presumably for AD integration). The key to this at the end would be to actually get a performance gain from the authentication itself, but I guess that would be something for the Samba list. -- Regards, Giles Coochey, CCNA, CCNAS NetSecSpec Ltd +44 (0) 7983 877438 http://www.coochey.net http://www.netsecspec.co.uk giles at coochey.net -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4941 bytes Desc: S/MIME Cryptographic Signature URL: From tss at iki.fi Tue Jul 3 00:50:57 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 00:50:57 +0300 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> Message-ID: <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> On 2.7.2012, at 19.12, Kaya Saman wrote: > what's really weird is that if I keep increasing the Cache TTL and > Cache size, the speed of transfer starts dropping. I think it may just be a coincidence that changing cache values appears to help, and the real reason maybe being just that Dovecot got restarted. Because if Outlook is using NTLM + winbind for authentication, the auth cache isn't used at all. From tss at iki.fi Tue Jul 3 00:55:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 00:55:35 +0300 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: References: Message-ID: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> On 2.7.2012, at 16.51, Dovecot user wrote: > imap-login: Aborted login (auth failed, 1 attempts): user=, > method=CRAM-MD5, rip=127.0.0.1, lip=127.0.0.1, TLS > Jul 2 15:19:13 mx > dovecot: auth-worker: mysql(localhost): Connected to database mail > Jul 2 > 15:19:15 mx dovecot: imap-login: Aborted login (auth failed, 1 > attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured > > > Can you help me please ? Isn't there anything more in the logs? It should at the very least say "password mismatch".. CRAM-MD5 authentication is a bit annoying to debug though. But I haven't touched its code for a long time, so it shouldn't have gotten broken. From tss at iki.fi Tue Jul 3 02:51:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 02:51:08 +0300 Subject: [Dovecot] Proxy config help please In-Reply-To: References: Message-ID: <19E902D9-7277-4585-AB65-A54193D9BAC3@iki.fi> On 30.6.2012, at 0.41, Zac Israel wrote: > # 2.0.19: /etc/dovecot/dovecot.conf .. > passdb { > args = proxy=proxy_always nopassword=y host=172.16.0.13 port=143 > proxy_timeout=5 starttls=y ssl=any-cert v2.0 has some problems with this. You should use v2.1 and use server name as the "host" value instead of IP. But anyway, the main problem is that you haven't specified ssl_ca setting that contains the accepted CA certificate. From tss at iki.fi Tue Jul 3 02:55:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 02:55:30 +0300 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <20120629182127.653130ctu3bfqxaf@mail.tu-chemnitz.de> References: <20120629182127.653130ctu3bfqxaf@mail.tu-chemnitz.de> Message-ID: <53B237A0-3A44-47DC-B41A-82CB5D174254@iki.fi> On 29.6.2012, at 19.21, Daniel Parthey wrote: > Jun 29 15:40:31 10.129.3.249 dovecot: doveadm(user31 at domain1.example.org): Error: user user31 at domain1.example.org: Error reading configuration: net_connect_unix(/var/run/dovecot/config) failed: Permission denied I've noticed a similar problem happening somewhat randomly, but I still haven't looked into why exactly it happens. Anyway the attached patch should fix this specific error, but I'm not sure if there isn't another one. Try and let me know? :) -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 449 bytes Desc: not available URL: From tss at iki.fi Tue Jul 3 03:08:08 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 03:08:08 +0300 Subject: [Dovecot] indexer-worker In-Reply-To: References: <1340865987.25551.67.camel@innu> Message-ID: <8EEA126F-A673-4E1D-820F-6227343BCAE8@iki.fi> On 28.6.2012, at 13.38, Wojciech Puchar wrote: >>> 29413 root 1 76 0 22820K 9204K kqread 1 0:17 5.86% indexer-worker >> >> It runs as root while not really doing anything, but when it starts >> accessing users' files it temporarily drops privileges. This is >> necessary if users have multiple different UIDs. > > to showed it with root privilege and 60% CPU load+disk I/O when doing text search over not yet indexed folder. Maybe your ps/kernel shows the process's "real uid" instead of the "effective uid". >> If you have only one UID e.g. vmail, you could set: >> > i'm not sure what you exactly mean. > > I have simplest possible config - mail accounts are unix accounts and mail is at Maildir Yes, so the problem is that if the same indexer-worker process wants to index mails to two different users, it can't drop root privileges completely because otherwise it couldn't switch between the users' UIDs. Anyway, it would be possible to drop the root privileges completely by servicing only a single indexing request and then stopping the process. This would be worse for performance of course, but it should still be an option. I'll probably add to v2.2. From tss at iki.fi Tue Jul 3 03:09:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 03:09:43 +0300 Subject: [Dovecot] Setting up mixed mbox and maildir In-Reply-To: <1340840055.2391.26.camel@amito> References: <1340742900.2495.14.camel@amito> <1340745560.2495.27.camel@amito> <5B14BB9D-490F-49F6-B647-D69F26DAE888@iki.fi> <1340840055.2391.26.camel@amito> Message-ID: On 28.6.2012, at 2.34, Jonathan Ryshpan wrote: > I am continuing to attempt to set up dovecot to keep its mail store in maildir > form while receiving it from an mbox, but without success. Dovecot reports the > error that it can't create the file ~/mail/.imap/INBOX (and also that it can't > chown it to user mail, not surprising since it doesn't exist). This seems > reasonable, since jonrysh (that is me) is not a member of the group mail. What > should be done next? Should I join the group mail? It seems that this should > not be necessary in general. The mail store is in ~/maildir, so what is the > function of the mbox ~/mail? > > Dovecot must be misconfigured, but it's not clear to a newbie like myself > what's wrong. Any advice would be appreciated. I have attached an extract > from maillog showing the errors (dovecot.log) dovecot reports (dovecot.log), > and the output of dovecot -n (dovecot-n). The log shows the URL for the explanation and the solution: http://wiki2.dovecot.org/Errors/ChgrpNoPerm From tss at iki.fi Tue Jul 3 03:31:41 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 03:31:41 +0300 Subject: [Dovecot] dict Panic after upgrade to 2.1.7 In-Reply-To: <201206241221.16044.ckubu@so36.net> References: <201206241221.16044.ckubu@so36.net> Message-ID: <7CEC4D79-FC7B-447E-9C1D-960B1D1606D2@iki.fi> On 24.6.2012, at 13.21, ckubu wrote: > dovecot log entries: > Jun 23 23:19:10 mx dovecot: dict: Panic: file driver-pgsql.c: line 84 > (driver_pgsql_set_state): assertion failed: (state == SQL_DB_STATE_BUSY || db- >> cur_result == NULL) This is clearly a bug, but I don't really see why it's happening. It would be helpful if you could get a gdb backtrace: http://dovecot.org/bugreport.html From tss at iki.fi Tue Jul 3 03:37:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 03:37:20 +0300 Subject: [Dovecot] Wrong headers in dovecot-crlf In-Reply-To: References: Message-ID: <4D3966F3-E268-4AC4-BFA5-06BA7A03ACAA@iki.fi> On 26.6.2012, at 21.55, Matthieu RAKOTOJAONA wrote: > I'm using the very good imaptest [0] tool to test my little imap server > implementation. I've tried to use the dovecot-crlf [1] file, but it looks like > there are some major issues : > > $ grep -n "In-Reply-To.*;" tests/data/dovecot-crlf > 479:In-Reply-To: <20020806175441.GA7148 at linux.taugt.net>; from > rueckert at informatik.uni-rostock.de on Tue, Aug 06, 2002 at 07:54:41PM +0200 Yes, those aren't valid RFC 2822 headers. I'm not entirely sure if they're valid RFC 822 headers. > As you can see, many of the "In-Reply-To" headers are polluted with some junk. > The situation is the same for many "Message-ID" headers. > > I don't know why they are here, but I think it's a mistake. I thought I would > let you know. They're there because those headers were sent originally to this list. I don't have any specific reason to modify the mails, and it might be better not to do it so that real world parsers can also figure out what to do with broken mails. From tss at iki.fi Tue Jul 3 03:38:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 03:38:24 +0300 Subject: [Dovecot] userdb errors after upgrading to 2.1 In-Reply-To: References: Message-ID: <53BB94C0-CE06-468D-BEC6-0F6641D103F5@iki.fi> On 26.6.2012, at 18.04, Andr? Rodier wrote: > I am using LDAP lookups, and virtual users with the same UID/GID. > > Everything was working fine before, but now, I have this error when I > try to send an email to a local account: > > -------------------------------------------------------- > Jun 26 15:46:52 lapetus dovecot: lmtp(24518): Error: user > user.test at indienet.com: Auth USER lookup failed > Jun 26 15:46:52 lapetus dovecot: auth: Error: > userdb(user.test at indienet.com,127.0.0.1): client doesn't have lookup > permissions for this user: userdb reply doesn't contain uid (change > userdb socket > -------------------------------------------------------- > > However, even if I set the permissions to 0666, I still have the same error. > > Can you point me in the right direction to fix this, please? Easiest fix is to set permissions to 0777. A better one would be to set them to 0600 and set the socket's UID to your virtual user's UID. From tss at iki.fi Tue Jul 3 03:51:05 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 03:51:05 +0300 Subject: [Dovecot] Additional passdb result status In-Reply-To: <1340570220.13783.23.camel@P7230> References: <1340570220.13783.23.camel@P7230> Message-ID: <636DE6FA-4C1E-43FA-9BC6-7BF90531ED54@iki.fi> On 24.6.2012, at 23.37, J?rgen Pabel wrote: > I am implementing a plugin (for the pop3/imap process) that requires > some data to provided from the authentication phase (a derivative of the > password). For that, I have now implemented a passdb plugin that > generates this data and I would like to "pass" this data down to the > mail process (pop3/imap) via extra_fields in the reply of the > authentication. The general idea is that my custom passdb plugin > calculates the data, sets the extra_field and returns some error > (authentication was not successful) so that the "real" passdb backend > can be invoked to "really" validate the authentication data. This is a bit related to another feature people have requested: Ability to merge data from multiple userdbs into a single reply. Perhaps the same could be done for passdbs. Also in my TODO is that master=yes passdb currently preserves userdb extra fields, but not passdb extra fields and that behavior probably isn't optimal. There is already passdb { pass=yes } setting for masterdbs. I guess the same could be used for non-masterdbs and also added to userdbs. > However, in auth_request_handle_passdb_callback() the extra_fields are > reseted unless the return code is PASSDB_RESULT_USER_DISABLED. But if > that return code is used then any following passdb's aren't invoked any > more - which makes sense with respect to user authenticiation. I would > therefore like to propose that some IGNORE/CONTINUE-status to be > introduced in auth/passdb.h, that would be handled in that extra_fields > and possible other values are not reseted in order to allow such > propagation of data from authentication process down to the mail process > (which could be extracted from the reply string by parsing it). Yes, extra_fields really need to get reset between passdb calls. Same for userdb_reply. > As a further implementation alternative (to the parsing of the reply > string), I also propose that some new "environment" item be introduced > (in auth_request) in order to allow such data passing in a generic > manner. Maybe a new permanent_extra_fields, which gets added as default to new passdb/userdb lookups. If the lookup has pass=yes, the result gets added to permanent_extra_fields. Although the code is beginning to have too many extra_fields variables. Maybe it would be possible to merge extra_fields, extra_cache_fields and userdb_reply into one array of structs: struct auth_field { const char *key, *value; bool cache_only; bool userdb; bool permanent; }; > I hope you consider my proposal to be reasonable. If desired, I could > implement this myself and provide a patch for merging (based on 2.0.x). > If my proposal is generally unfavored, it would be great if any > alternative approaches for my situation were suggested. Thanks. I could add such a patch to v2.2. From tss at iki.fi Tue Jul 3 05:22:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 03 Jul 2012 05:22:25 +0300 Subject: [Dovecot] v2.1.8 released Message-ID: <1341282145.19913.26.camel@hurina> http://dovecot.org/releases/2.1/dovecot-2.1.8.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.8.tar.gz.sig Most importantly fixes a pretty bad behavior of sending mailbox names as UTF-8 instead of mUTF-7 as required by IMAP RFC. I'm surprised nobody hadn't noticed this in v2.1.x tree before yesterday. I guess some clients might have translated the name to UTF-8 anyway and not noticed it, while others would have silently ignored the problem of not seeing STATUS updates. + pop3c: Added pop3c_master_user setting. - imap: Mailbox names were accidentally sent as UTF-8 instead of mUTF-7 in previous v2.1.x releases for STATUS, MYRIGHTS and GETQUOTAROOT commands. - lmtp proxy: Don't timeout connections too early when mail has a lot of RCPT TOs. - director: Don't crash if the director is working alone. - shared mailboxes: Avoid doing "@domain" userdb lookups. - doveadm: Fixed crash with proxying some commands. - fts-squat: Fixed handling multiple SEARCH parameters. - imapc: Fixed a crash when message had more than 8 keywords. - imapc: Don't crash on APPEND/COPY if server doesn't support UIDPLUS. From dovecot at e-solutions.re Tue Jul 3 07:37:35 2012 From: dovecot at e-solutions.re (Dovecot user) Date: Tue, 03 Jul 2012 08:37:35 +0400 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> References: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> Message-ID: <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> There's no way to troubleshoot this error ? I use the same config file for dovecot 2.0.13, all works. Perhaps in 2.0.17, something is missing in my config file, or there's new keyword... ?? 'dovecot -n' give me : auth_mechanisms = plain login digest-md5 cram-md5 apop base_dir = /var/dovecot/ first_valid_uid = 1000 mail_location = maildir:/var/mailserv/mail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /etc/dovecot-sql.conf driver = sql } plugin { antispam_mail_notspam = --ham antispam_mail_sendmail = /usr/local/bin/sa-learn antispam_mail_sendmail_args = --username=%u antispam_mail_spam = --spam antispam_mail_tmpdir = /tmp antispam_signature = X-Spam-Flag antispam_signature_missing = move antispam_spam = SPAM;Spam;spam;Junk;junk antispam_trash = trash;Trash;Deleted Items; Deleted Messages autocreate = Trash autocreate2 = Spam autocreate3 = Sent autocreate4 = Drafts autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Sent autosubscribe4 = Drafts quota = maildir quota_rule = *:storage=5G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% /usr/local/bin/quota-warning.sh 95 quota_warning2 = storage=80%% /usr/local/bin/quota-warning.sh 80 sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap sieve pop3 service auth { unix_listener /var/run/dovecot-auth-master { group = _dovecot mode = 0666 user = _dovecot } unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } } service imap-login { service_count = 0 vsz_limit = 64 M } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service pop3-login { service_count = 0 vsz_limit = 64 M } ssl_cert = On 2.7.2012, at 16.51, Dovecot user wrote: > >> imap-login: Aborted login (auth failed, 1 attempts): user=, method=CRAM-MD5, rip=127.0.0.1, lip=127.0.0.1, TLS Jul 2 15:19:13 mx dovecot: auth-worker: mysql(localhost): Connected to database mail Jul 2 15:19:15 mx dovecot: imap-login: Aborted login (auth failed, 1 attempts): user=, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Can you help me please ? > > Isn't there anything more in the logs? It should at the very least say "password mismatch".. CRAM-MD5 authentication is a bit annoying to debug though. But I haven't touched its code for a long time, so it shouldn't have gotten broken. From tss at iki.fi Tue Jul 3 07:54:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 07:54:02 +0300 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> References: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> Message-ID: <336010E0-D487-4CA5-82B1-FC255048935D@iki.fi> On 3.7.2012, at 7.37, Dovecot user wrote: > There's no way to troubleshoot this error ? Set auth_debug_passwords=yes and show all of the log entries from a failed login. Preferably use a test password so it won't be exposed. :) > I use the same config > file for dovecot 2.0.13, all works. > Perhaps in 2.0.17, something is > missing in my config file, or there's new keyword... ?? No, the configuration is identical between those version. The CRAM-MD5 code is also identical between those versions. I guess there could be some other seemingly unrelated change that might have broken it for some reason. From dovecot at e-solutions.re Tue Jul 3 08:00:33 2012 From: dovecot at e-solutions.re (Dovecot user) Date: Tue, 03 Jul 2012 09:00:33 +0400 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: <336010E0-D487-4CA5-82B1-FC255048935D@iki.fi> References: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> <336010E0-D487-4CA5-82B1-FC255048935D@iki.fi> Message-ID: <8ac95e69cbdefa2a2ceb33f9cd9ef5ef@e-solutions.re> I tried to do the same 'dovecot -n' (dovecot 2.0.13, same config file) : # 2.0.13: /etc/dovecot/dovecot.conf # OS: OpenBSD 5.0 i386 ffs auth_mechanisms = plain login digest-md5 cram-md5 apop base_dir = /var/dovecot/ default_internal_user = _dovecot default_login_user = _dovenull first_valid_uid = 1000 mail_location = maildir:/var/mailserv/mail/%d/%n With dovecot 2.0.17 : # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: OpenBSD 5.1 i386 ffs auth_mechanisms = plain login digest-md5 cram-md5 apop base_dir = /var/dovecot/ first_valid_uid = 1000 mail_location = maildir:/var/mailserv/mail/%d/%n Is it normal that there are no lines : "default_internal_user = _dovecot" & "default_login_user = _dovenull" ?? I will try "auth_debug_passwords=yes", keep you informed. Again, thank you very much. -- Wesley On 03.07.2012 08:54, Timo Sirainen wrote: > On 3.7.2012, at 7.37, Dovecot user wrote: > >> There's no way to troubleshoot this error ? > > Set auth_debug_passwords=yes and show all of the log entries from a failed login. Preferably use a test password so it won't be exposed. :) > >> I use the same config file for dovecot 2.0.13, all works. Perhaps in 2.0.17, something is missing in my config file, or there's new keyword... ?? > > No, the configuration is identical between those version. The CRAM-MD5 code is also identical between those versions. I guess there could be some other seemingly unrelated change that might have broken it for some reason. From tss at iki.fi Tue Jul 3 08:20:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 08:20:13 +0300 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: <8ac95e69cbdefa2a2ceb33f9cd9ef5ef@e-solutions.re> References: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> <336010E0-D487-4CA5-82B1-FC255048935D@iki.fi> <8ac95e69cbdefa2a2ceb33f9cd9ef5ef@e-solutions.re> Message-ID: <8414D624-13CB-476A-8EE1-1DC7DA4B1929@iki.fi> On 3.7.2012, at 8.00, Dovecot user wrote: > I tried to do the same 'dovecot -n' (dovecot 2.0.13, same config > file) : Oh, are you saying that the difference isn't just the Dovecot version, but that they are completely different systems with (somewhat) different settings also? That's much more likely the problem than the version number. > Is it normal that there are no lines : > "default_internal_user = _dovecot" & "default_login_user = _dovenull" Normal enough. It's then using the Dovecot's defaults which are "dovecot" and "dovenull". > I will try "auth_debug_passwords=yes", keep you informed. Look at this output in your 2.0.13 setup and in 2.0.17 setup and see what the differences are. From dovecot at e-solutions.re Tue Jul 3 08:34:59 2012 From: dovecot at e-solutions.re (Dovecot user) Date: Tue, 03 Jul 2012 09:34:59 +0400 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: <8414D624-13CB-476A-8EE1-1DC7DA4B1929@iki.fi> References: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> <336010E0-D487-4CA5-82B1-FC255048935D@iki.fi> <8ac95e69cbdefa2a2ceb33f9cd9ef5ef@e-solutions.re> <8414D624-13CB-476A-8EE1-1DC7DA4B1929@iki.fi> Message-ID: <032171a18114ac267b37172cc563876a@e-solutions.re> i just get this error : dovecot: imap-login: Aborted login (auth failed, 1 attempts): method=DIGEST-MD5, rip=127.0.0.1, lip=127.0.0.1, secured root at mx:/var/log # date IMAP Error: Login failed for testing at mydomain.net from 192.168.0.92. Authentication failed. in /var/www/webmail/roundcubemail-0.7.2/program/include/rcube_imap.php on line 205 (POST /webmail/?_task=login&_action=login) On 03.07.2012 09:20, Timo Sirainen wrote: > On 3.7.2012, at 8.00, Dovecot user wrote: > >> I tried to do the same 'dovecot -n' (dovecot 2.0.13, same config file) : > > Oh, are you saying that the difference isn't just the Dovecot version, but that they are completely different systems with (somewhat) different settings also? That's much more likely the problem than the version number. > >> Is it normal that there are no lines : "default_internal_user = _dovecot" & "default_login_user = _dovenull" > > Normal enough. It's then using the Dovecot's defaults which are "dovecot" and "dovenull". From dovecot at asheesh.org Tue Jul 3 08:53:35 2012 From: dovecot at asheesh.org (Asheesh Laroia) Date: Tue, 03 Jul 2012 01:53:35 -0400 Subject: [Dovecot] Dovecot shared library to replace libc-client Message-ID: <1341293710-sup-3541@rose.makesad.us> Hi all Dovecotters, Lots of programs (e.g, PHP) link to a library called c-client, which is a derivative of the original IMAP implementation, UW IMAP. (For those new to UW IMAP, read here: http://www.washington.edu/imap/ ) UW IMAP is not in as much of active development as it used to be, so I am curious: I see there is a dovecot shared library. I haven't looked into the details, but here are things I'm interested in: 1. Replacing libc-client's use as a client library In Debian, for example, you can search for packages that depend on this library as a client. It looks like this: $ ~ apt-cache rdepends libc-client2007e libc-client2007e Reverse Depends: uw-mailutils uw-imapd libc-client2007e-dev ipopd prayer php5-imap mailsync libmail-cclient-perl asterisk aolserver4-nsimap uw-mailutils libc-client2007e-dev prayer php5-imap mailsync libmail-cclient-perl asterisk-voicemail-imapstorage Let's say, hypothetically, we wanted to switch php5-imap to using Dovecot where it currently uses libc-client. I envision creating libdovecot-c-client-alike that is a set of headers and a library that is API-compatible with (at least a subset of) c-client. You can call that a "compatibility shim." Then e.g. php5-imap could be given the path to those headers and the corresponding libdovecot-c-client-alike library, and when it thinks it is linking to c-client, it could instead link to the libdovecot-c-client-alike. This might be convenient if you want to limit how much of a public API is presented by the current "dovecot.so" that gets installed in e.g. /usr/lib/dovecot/. The compatiblity shim could have a small API, and if you don't want provide ABI guarantees within dovecot.so, the shim could dlopen() dovecot.so rather than link to it. 2. Use of Dovecot shared library within alpine, embedding the imapd Right now, the mail client "alpine" embeds a copy of the UW IMAP source. It uses this when accessing local mail spools, for example. If Dovecot's IMAPd were available as a shared library, perhaps with a c-client-like API, (although not necessarily -- it would be feasible to upgrade alpine to a different API), then alpine could use Dovecot's mail drivers directly. So, those two are the dream. Timo and others, what are your thoughts? Thanks for reading this far! -- Asheesh. P.S. The radical request #2 would be an excellent, decisive way to end a sad thread in the Debian bug tracker about Alpine + Maildir: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=405762 . From wojtek at wojtek.tensor.gdynia.pl Tue Jul 3 08:54:56 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 3 Jul 2012 07:54:56 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF1E099.5010908@schetterer.org> References: <4FF1C51C.8060509@coochey.net> <4FF1D138.6050801@gedalya.net> <4FF1E099.5010908@schetterer.org> Message-ID: > Outlook is the client of Exchange , it can do smtp,imap,pop3 too > > its sold as a solution, with os , server, auth system ,client , support > people etc and that solution - as a common example - doesn't work. > There is no reason for M$ to make Qutlook fit in a perfect > imap client cause this would goal in less reason for exchange as well as it's own product. But it is by design, as more money can ge acquired by constantly servicing non working product. > Nevertheless if you have good admins and money to buy licences > and good support, outlook and exchange are good company solutions for > intranet mail and groupware ( like calendering etc ) Show me working case. i would like to see it :) Still - it have nothing to Dovecot and Dovecot, no matter how great would be, will not improve it. "Tweaks" can overcome some bugs or deficiences but not whole product. From wojtek at wojtek.tensor.gdynia.pl Tue Jul 3 08:56:50 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 3 Jul 2012 07:56:50 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> Message-ID: finally some clear answer :) Trying dovecot to improve outlook is like using gold to improve shit. No matter how much gold is used, it will still stink. I cannot understand that people. On Tue, 3 Jul 2012, Timo Sirainen wrote: > On 2.7.2012, at 19.12, Kaya Saman wrote: > >> what's really weird is that if I keep increasing the Cache TTL and >> Cache size, the speed of transfer starts dropping. > > I think it may just be a coincidence that changing cache values appears to help, and the real reason maybe being just that Dovecot got restarted. Because if Outlook is using NTLM + winbind for authentication, the auth cache isn't used at all. > > > From wojtek at wojtek.tensor.gdynia.pl Tue Jul 3 08:58:06 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 3 Jul 2012 07:58:06 +0200 (CEST) Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <20120702161429.5cd29802@scorpio> References: <4FF1C41F.4060200@Media-Brokers.com> <20120702161429.5cd29802@scorpio> Message-ID: >> Many companies require Outlook, and the fact is, as an EXCHANGE >> client, Outlook works extremely well. I agree that as a standalone Show me this. I actually migrated many places OUT of this crap because it doesn't work, with great success. From tss at iki.fi Tue Jul 3 09:12:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 09:12:36 +0300 Subject: [Dovecot] Dovecot shared library to replace libc-client In-Reply-To: <1341293710-sup-3541@rose.makesad.us> References: <1341293710-sup-3541@rose.makesad.us> Message-ID: <19DB0A1D-AB49-4108-9E98-B85E59DF2050@iki.fi> On 3.7.2012, at 8.53, Asheesh Laroia wrote: > I see there is a dovecot shared library. I haven't looked into the > details, but here are things I'm interested in: > > 1. Replacing libc-client's use as a client library .. > I envision creating libdovecot-c-client-alike that is a set of headers > and a library that is API-compatible with (at least a subset of) > c-client. You can call that a "compatibility shim." Then e.g. php5-imap > could be given the path to those headers and the corresponding > libdovecot-c-client-alike library, and when it thinks it is linking to > c-client, it could instead link to the libdovecot-c-client-alike. Yeah, that's a possibility. Although Dovecot's libraries are still more about the server side stuff than client side stuff, so it's possible that there are many important missing things. Also libc-client is commonly used to do IMAP access and Dovecot's imapc backend is still lacking quite a lot of that functionality. > This might be convenient if you want to limit how much of a public API > is presented by the current "dovecot.so" that gets installed in e.g. > /usr/lib/dovecot/. The compatiblity shim could have a small API, and if > you don't want provide ABI guarantees within dovecot.so, the shim could > dlopen() dovecot.so rather than link to it. I'm still not ready to give ABI or even API guarantees to libdovecot.. There are still several important large changes to do and I don't really want to keep a ton of ugly backwards compatibility stuff just for external users of the library. Also another potential problem is that libdovecot.so doesn't use a global namespace prefix for all of its functions, so linking it with php could cause symbol name conflicts (especially md5_*, sha1_* and such could cause trouble, like they already have caused with libmysql). Once Dovecot becomes more "finished" (a few years?) I could consider API/ABI guarantees.. Of course nothing prevents anyone else from distributing a (patched) libdovecot already that actually does give some ABI guarantees. I just don't want to spend time on it. And v2.1 -> v2.2 -> v2.3 etc. transitions are going to be large changes. > 2. Use of Dovecot shared library within alpine, embedding the imapd > > Right now, the mail client "alpine" embeds a copy of the UW IMAP > source. It uses this when accessing local mail spools, for example. > > If Dovecot's IMAPd were available as a shared library, perhaps with a > c-client-like API, (although not necessarily -- it would be feasible to > upgrade alpine to a different API), then alpine could use Dovecot's > mail drivers directly. I wonder if it would make any sense to for Alpine not use libdovecot API directly but rather talk IMAP protocol to Dovecot code (maybe running in a separate process)? The Dovecot configuration could be passed pretty easily from Alpine code without requiring any extra config files. From tss at iki.fi Tue Jul 3 09:13:39 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 09:13:39 +0300 Subject: [Dovecot] Authentication failed (migrate from 2.0.13 to 2.0.17) In-Reply-To: <032171a18114ac267b37172cc563876a@e-solutions.re> References: <5ABB1C7C-7BA5-467D-92C7-01AB97677712@iki.fi> <356665ff787d48ea12d645f96dcea0ff@e-solutions.re> <336010E0-D487-4CA5-82B1-FC255048935D@iki.fi> <8ac95e69cbdefa2a2ceb33f9cd9ef5ef@e-solutions.re> <8414D624-13CB-476A-8EE1-1DC7DA4B1929@iki.fi> <032171a18114ac267b37172cc563876a@e-solutions.re> Message-ID: <4B350859-C06F-408B-8719-5B4D7959BC92@iki.fi> With auth_debug_passwords=yes you'll get a lot more in the log. If you don't see, you're not looking into the correct file. Make sure you look into the debug log as shown by "doveadm log find". On 3.7.2012, at 8.34, Dovecot user wrote: > > > i just get this error : > dovecot: imap-login: Aborted login (auth > failed, 1 attempts): method=DIGEST-MD5, rip=127.0.0.1, lip=127.0.0.1, > secured > root at mx:/var/log # date > > IMAP Error: Login failed for > testing at mydomain.net from 192.168.0.92. Authentication failed. in > /var/www/webmail/roundcubemail-0.7.2/program/include/rcube_imap.php on > line 205 (POST /webmail/?_task=login&_action=login) > > On 03.07.2012 > 09:20, Timo Sirainen wrote: > >> On 3.7.2012, at 8.00, Dovecot user > wrote: >> >>> I tried to do the same 'dovecot -n' (dovecot 2.0.13, same > config file) : >> >> Oh, are you saying that the difference isn't just > the Dovecot version, but that they are completely different systems with > (somewhat) different settings also? That's much more likely the problem > than the version number. >> >>> Is it normal that there are no lines : > "default_internal_user = _dovecot" & "default_login_user = _dovenull" >> > >> Normal enough. It's then using the Dovecot's defaults which are > "dovecot" and "dovenull". > From dovecot at asheesh.org Tue Jul 3 09:18:46 2012 From: dovecot at asheesh.org (Asheesh Laroia) Date: Tue, 3 Jul 2012 02:18:46 -0400 (EDT) Subject: [Dovecot] Dovecot shared library to replace libc-client In-Reply-To: <19DB0A1D-AB49-4108-9E98-B85E59DF2050@iki.fi> References: <1341293710-sup-3541@rose.makesad.us> <19DB0A1D-AB49-4108-9E98-B85E59DF2050@iki.fi> Message-ID: On Tue, 3 Jul 2012, Timo Sirainen wrote: > On 3.7.2012, at 8.53, Asheesh Laroia wrote: > >> I see there is a dovecot shared library. I haven't looked into the >> details, but here are things I'm interested in: >> >> 1. Replacing libc-client's use as a client library > .. >> I envision creating libdovecot-c-client-alike that is a set of headers >> and a library that is API-compatible with (at least a subset of) >> c-client. You can call that a "compatibility shim." Then e.g. php5-imap >> could be given the path to those headers and the corresponding >> libdovecot-c-client-alike library, and when it thinks it is linking to >> c-client, it could instead link to the libdovecot-c-client-alike. > > Yeah, that's a possibility. Although Dovecot's libraries are still more > about the server side stuff than client side stuff, so it's possible > that there are many important missing things. Also libc-client is > commonly used to do IMAP access and Dovecot's imapc backend is still > lacking quite a lot of that functionality. Timo, I appreciate the super speedy response! >> This might be convenient if you want to limit how much of a public API >> is presented by the current "dovecot.so" that gets installed in e.g. >> /usr/lib/dovecot/. The compatiblity shim could have a small API, and if >> you don't want provide ABI guarantees within dovecot.so, the shim could >> dlopen() dovecot.so rather than link to it. > > I'm still not ready to give ABI or even API guarantees to libdovecot.. > There are still several important large changes to do and I don't really > want to keep a ton of ugly backwards compatibility stuff just for > external users of the library. Also another potential problem is that > libdovecot.so doesn't use a global namespace prefix for all of its > functions, so linking it with php could cause symbol name conflicts > (especially md5_*, sha1_* and such could cause trouble, like they > already have caused with libmysql). Yeah, I totally understand your desire to not make backwards compatiblity a goal of the project. Interesting point about the global namespace prefix. Is this something you'd be willing to reconsider, and start using a global namespace prefix? > Once Dovecot becomes more "finished" (a few years?) I could consider > API/ABI guarantees.. Of course nothing prevents anyone else from > distributing a (patched) libdovecot already that actually does give some > ABI guarantees. I just don't want to spend time on it. And v2.1 -> v2.2 > -> v2.3 etc. transitions are going to be large changes. Yeah -- what I think is the most sensible, at the moment, is to distribute a small shim that has reasonably-tight dependencies to dovecot itself, and so when you upgrade dovecot, you probably have to upgrade the shim. So it proxies away the instability in dovecot, and provides a small, stable API/ABI. That's something that it seems you might not be interested in, but I wonder if I can convince you otherwise. If not, I might try convincing others to write it, but I'm hoping you might since you are so great! (-: >> 2. Use of Dovecot shared library within alpine, embedding the imapd >> >> Right now, the mail client "alpine" embeds a copy of the UW IMAP >> source. It uses this when accessing local mail spools, for example. >> >> If Dovecot's IMAPd were available as a shared library, perhaps with a >> c-client-like API, (although not necessarily -- it would be feasible to >> upgrade alpine to a different API), then alpine could use Dovecot's >> mail drivers directly. > > I wonder if it would make any sense to for Alpine not use libdovecot API > directly but rather talk IMAP protocol to Dovecot code (maybe running in > a separate process)? The Dovecot configuration could be passed pretty > easily from Alpine code without requiring any extra config files. That's my fallback plan at the moment, yeah. It seems like more work, though, but it has some serious tidiness possibly going for it. -- Asheesh. From tss at iki.fi Tue Jul 3 09:37:45 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 09:37:45 +0300 Subject: [Dovecot] Dovecot shared library to replace libc-client In-Reply-To: References: <1341293710-sup-3541@rose.makesad.us> <19DB0A1D-AB49-4108-9E98-B85E59DF2050@iki.fi> Message-ID: <91ACF408-ED63-4DFA-9E36-75C2B7F23342@iki.fi> On 3.7.2012, at 9.18, Asheesh Laroia wrote: >> I'm still not ready to give ABI or even API guarantees to libdovecot.. There are still several important large changes to do and I don't really want to keep a ton of ugly backwards compatibility stuff just for external users of the library. Also another potential problem is that libdovecot.so doesn't use a global namespace prefix for all of its functions, so linking it with php could cause symbol name conflicts (especially md5_*, sha1_* and such could cause trouble, like they already have caused with libmysql). > > Yeah, I totally understand your desire to not make backwards compatiblity a goal of the project. > > Interesting point about the global namespace prefix. Is this something you'd be willing to reconsider, and start using a global namespace prefix? Dovecot in any case will have many different prefixes (I don't want to change ALL functions to begin with the same one), but there is one that is somewhat commonly used already: "i_" which originally indicated "irssi library" :) dovecot-2.1/src/lib% grep '\bi_' *.h|wc -l 175 Much of the rest of the functions in lib/ could be prefixed with "i_" I guess. But there are several annoying problems, such as t_ prefix is also commonly used in many places and I don't want to remove them (but they probably won't be a real problem either). Then there's i_stream_*() for input streams vs. o_stream_*() for output streams. But for some of the clearly unproblematic ones the i_ prefix could be added, like for md5/sha/etc small and potentially conflicting functions. Also I think GNU ld supports some way of hiding all but the explicitly wanted symbols from libraries, which would allow libdovecot to use all of its functions internally without the danger of them conflicting with outside users. But I'm not entirely sure how that works, maybe it needs to be done at linking stage which won't be good for Dovecot binaries that link with libdovecot. >> Once Dovecot becomes more "finished" (a few years?) I could consider API/ABI guarantees.. Of course nothing prevents anyone else from distributing a (patched) libdovecot already that actually does give some ABI guarantees. I just don't want to spend time on it. And v2.1 -> v2.2 -> v2.3 etc. transitions are going to be large changes. > > Yeah -- what I think is the most sensible, at the moment, is to distribute a small shim that has reasonably-tight dependencies to dovecot itself, and so when you upgrade dovecot, you probably have to upgrade the shim. So it proxies away the instability in dovecot, and provides a small, stable API/ABI. > > That's something that it seems you might not be interested in, but I wonder if I can convince you otherwise. > > If not, I might try convincing others to write it, but I'm hoping you might since you are so great! (-: The best I can do is that at some point in future (might still be 1-2 years) I want to provide scripting language bindings for Dovecot APIs, maybe even allow creating plugins with non-C languages. Those APIs would then pretty much have to be frozen so that they don't break. Before that I need to do a few more larger API breakages to support some missing things. >> I wonder if it would make any sense to for Alpine not use libdovecot API directly but rather talk IMAP protocol to Dovecot code (maybe running in a separate process)? The Dovecot configuration could be passed pretty easily from Alpine code without requiring any extra config files. > > That's my fallback plan at the moment, yeah. It seems like more work, though, but it has some serious tidiness possibly going for it. You could have Alpine keep the libc-client code and simply use it to connect to local Dovecot via IMAP protocol. All of the mailbox access would then go via Dovecot. I don't think it would be much work, mainly you'd need to create "libc-client mail storage configuration" -> "Dovecot settings in key=value format", fork a new process, put the settings to environment, exec imap and that's about it. From kayasaman at gmail.com Tue Jul 3 09:38:22 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 07:38:22 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> Message-ID: On Mon, Jul 2, 2012 at 10:50 PM, Timo Sirainen wrote: > On 2.7.2012, at 19.12, Kaya Saman wrote: > >> what's really weird is that if I keep increasing the Cache TTL and >> Cache size, the speed of transfer starts dropping. > > I think it may just be a coincidence that changing cache values appears to help, and the real reason maybe being just that Dovecot got restarted. Because if Outlook is using NTLM + winbind for authentication, the auth cache isn't used at all. > Thanks Giles and Timo :-) So if I look at a different authentication mechanism say LDAP would it improve performance? There is currently a local AD in the network which I think piggy-backs off the parent companies AD for mail. If this is guarunteed to improve performance I don't mind taking the time to learn LDAP integration; though I must say it took me quite a while, several months in fact to learn NTLM and get it working - with very little or no help from anyone on any associated list :-( . Regards, Kaya From tss at iki.fi Tue Jul 3 09:46:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 3 Jul 2012 09:46:18 +0300 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> Message-ID: <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> On 3.7.2012, at 9.38, Kaya Saman wrote: > So if I look at a different authentication mechanism say LDAP would it > improve performance? I doubt authentication has anything to do with why Outlook downloads mails slowly. But you could configure Outlook to use plaintext authentication instead of NTLM authentication to see if it makes a difference. No need to change anything on Dovecot side then. From kayasaman at gmail.com Tue Jul 3 09:58:08 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 07:58:08 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> Message-ID: On Tue, Jul 3, 2012 at 7:46 AM, Timo Sirainen wrote: > On 3.7.2012, at 9.38, Kaya Saman wrote: > >> So if I look at a different authentication mechanism say LDAP would it >> improve performance? > > I doubt authentication has anything to do with why Outlook downloads mails slowly. > > But you could configure Outlook to use plaintext authentication instead of NTLM authentication to see if it makes a difference. No need to change anything on Dovecot side then. > I've just had a look and I don't think Outlook 2010 has that option.... ?? It might be that M$ decided to use auto authentication/negotiation as to take away the hastle from potentially confused end users? This is frustrating, using Thunderbird this setup works really well however, my organization **requires** Outlook and the only contectivity is to a remotely managed Exchange server (being connected to over VPN to another country) and no **standard** protocols used, just **Exchange** meaning I can't even connect to the server using a 'normal' client. :-S Am totally lost now! - BUMP! Regards, Kaya From role.Dovecot-Readers at JLAssocs.com Tue Jul 3 09:59:52 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Tue, 3 Jul 2012 07:59:52 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> Message-ID: <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> On 3 Jul 2012, at 07:46, Timo Sirainen wrote: > On 3.7.2012, at 9.38, Kaya Saman wrote: > >> So if I look at a different authentication mechanism say LDAP would it >> improve performance? > > I doubt authentication has anything to do with why Outlook downloads mails slowly. > > But you could configure Outlook to use plaintext authentication instead of NTLM authentication to see if it makes a difference. No need to change anything on Dovecot side then. It's a bit of a random tuppenyworth, but all my experience of slow Outlook clients seems to be local mail store sync work, perhaps garbage-collecting / defragmenting or something, but not actually getting the emails themselves . . I have one particular client who reported issues yesterday as it happens -- all versions of Windows from XP thru Win7 running mostly older Outlook but a couple of 2010 clients -- and one particular user, logged in on only one particular workstation (Win7 & 2010 as it happens) experiences _colossal_ delays in waiting for mail to open or respond at times, and yet any other user, or moving to another machine, it's all swift and fine. That smacks of a local desktop cache problem to me... All on the LAN, as well, no slow connections. As I say, just 0.02 -- may not be overly relevant, but my instinct is that local storage with Outlook has significant possibility for issues. J. From arne at drlinux.no Tue Jul 3 10:07:41 2012 From: arne at drlinux.no (Arne K. Haaje) Date: Tue, 03 Jul 2012 09:07:41 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> Message-ID: <4FF29A3D.8030300@drlinux.no> Den 03.07.2012 08:58, skrev Kaya Saman: > On Tue, Jul 3, 2012 at 7:46 AM, Timo Sirainen wrote: >> On 3.7.2012, at 9.38, Kaya Saman wrote: >> >>> So if I look at a different authentication mechanism say LDAP would it >>> improve performance? >> >> I doubt authentication has anything to do with why Outlook downloads mails slowly. >> >> But you could configure Outlook to use plaintext authentication instead of NTLM authentication to see if it makes a difference. No need to change anything on Dovecot side then. >> > > I've just had a look and I don't think Outlook 2010 has that option.... ?? I belive there is a checkbox there called something like "Use secure authentication - PKA(?)". Uncheck it, and you should have plaintext. Arne -- Arne K. Haaje http://www.drlinux.no/ ::: arne at drlinux.no LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb From kayasaman at gmail.com Tue Jul 3 10:09:42 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 08:09:42 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF29A3D.8030300@drlinux.no> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <4FF29A3D.8030300@drlinux.no> Message-ID: On Tue, Jul 3, 2012 at 8:07 AM, Arne K. Haaje wrote: > Den 03.07.2012 08:58, skrev Kaya Saman: > >> On Tue, Jul 3, 2012 at 7:46 AM, Timo Sirainen wrote: >>> >>> On 3.7.2012, at 9.38, Kaya Saman wrote: >>> >>>> So if I look at a different authentication mechanism say LDAP would it >>>> improve performance? >>> >>> >>> I doubt authentication has anything to do with why Outlook downloads >>> mails slowly. >>> >>> But you could configure Outlook to use plaintext authentication instead >>> of NTLM authentication to see if it makes a difference. No need to change >>> anything on Dovecot side then. >>> >> >> I've just had a look and I don't think Outlook 2010 has that option.... ?? > > > > I belive there is a checkbox there called something like "Use secure > authentication - PKA(?)". Uncheck it, and you should have plaintext. > > Arne > > -- > Arne K. Haaje > http://www.drlinux.no/ ::: arne at drlinux.no > LinkedIn: http://no.linkedin.com/pub/arne-haaje/27/189/bb > > Thanks for that, however it wasn't checked initially. Regards, Kaya From kayasaman at gmail.com Tue Jul 3 10:12:18 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 08:12:18 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> Message-ID: On Tue, Jul 3, 2012 at 7:59 AM, J E Lyon wrote: > On 3 Jul 2012, at 07:46, Timo Sirainen wrote: > >> On 3.7.2012, at 9.38, Kaya Saman wrote: >> >>> So if I look at a different authentication mechanism say LDAP would it >>> improve performance? >> >> I doubt authentication has anything to do with why Outlook downloads mails slowly. >> >> But you could configure Outlook to use plaintext authentication instead of NTLM authentication to see if it makes a difference. No need to change anything on Dovecot side then. > > It's a bit of a random tuppenyworth, but all my experience of slow Outlook clients seems to be local mail store sync work, perhaps garbage-collecting / defragmenting or something, but not actually getting the emails themselves . . > > I have one particular client who reported issues yesterday as it happens -- all versions of Windows from XP thru Win7 running mostly older Outlook but a couple of 2010 clients -- and one particular user, logged in on only one particular workstation (Win7 & 2010 as it happens) experiences _colossal_ delays in waiting for mail to open or respond at times, and yet any other user, or moving to another machine, it's all swift and fine. > > That smacks of a local desktop cache problem to me... All on the LAN, as well, no slow connections. > > As I say, just 0.02 -- may not be overly relevant, but my instinct is that local storage with Outlook has significant possibility for issues. > > J. Hmm... interesting point and had I been using a 'standard' filesystem type I would have to agree. However this is a clean server with plenty of space left on the pool allocated for mail and it's additionally using ZFS too. The point is that I am monitoring using nload as well as other things and the maximum bandwidth being got with Outlook is a few Mbps burst, average 50kbps; while with T-Bird I get way over 130Mbps??? Regards, Kaya From role.Dovecot-Readers at jlassocs.com Tue Jul 3 10:18:22 2012 From: role.Dovecot-Readers at jlassocs.com (J E Lyon) Date: Tue, 3 Jul 2012 08:18:22 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> Message-ID: <3DD05DEA-7DC6-412C-AC89-B745546BE8AD@jlassocs.com> On 3 Jul 2012, at 08:12, Kaya Saman wrote: > On Tue, Jul 3, 2012 at 7:59 AM, J E Lyon > wrote: >> On 3 Jul 2012, at 07:46, Timo Sirainen wrote: >> >>> On 3.7.2012, at 9.38, Kaya Saman wrote: >>> >>>> So if I look at a different authentication mechanism say LDAP would it >>>> improve performance? >>> >>> I doubt authentication has anything to do with why Outlook downloads mails slowly. >>> >>> But you could configure Outlook to use plaintext authentication instead of NTLM authentication to see if it makes a difference. No need to change anything on Dovecot side then. >> >> It's a bit of a random tuppenyworth, but all my experience of slow Outlook clients seems to be local mail store sync work, perhaps garbage-collecting / defragmenting or something, but not actually getting the emails themselves . . >> >> I have one particular client who reported issues yesterday as it happens -- all versions of Windows from XP thru Win7 running mostly older Outlook but a couple of 2010 clients -- and one particular user, logged in on only one particular workstation (Win7 & 2010 as it happens) experiences _colossal_ delays in waiting for mail to open or respond at times, and yet any other user, or moving to another machine, it's all swift and fine. >> >> That smacks of a local desktop cache problem to me... All on the LAN, as well, no slow connections. >> >> As I say, just 0.02 -- may not be overly relevant, but my instinct is that local storage with Outlook has significant possibility for issues. >> >> J. > > Hmm... interesting point and had I been using a 'standard' filesystem > type I would have to agree. > > > However this is a clean server with plenty of space left on the pool > allocated for mail and it's additionally using ZFS too. > > > The point is that I am monitoring using nload as well as other things > and the maximum bandwidth being got with Outlook is a few Mbps burst, > average 50kbps; while with T-Bird I get way over 130Mbps??? Oh, I may have come into the thread a little late and missed some details :) All the same, the customer I was mentioning has a fairly newish machine, though nothing fancy filesystem-wise. Sounds like you're monitoring what's actually happening anyway. Having said that, I've _always_ seen Microsoft network activity fall way below other protocols, so it might not be so surprising -- and if the local store is busy shuffling every message each time a new one is added, that would explain a lower load on the network while the local client was busy chasing its own tail. Do you mean that the clean install has very little email saved locally yet, and that Dovecot has little content for it to retrieve? So, there surely couldn't be any local file thrashing, there... J. From dirk.jahnke-zumbusch at desy.de Tue Jul 3 10:26:14 2012 From: dirk.jahnke-zumbusch at desy.de (Jahnke-Zumbusch, Dirk) Date: Tue, 3 Jul 2012 09:26:14 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are thereany tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it><4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net><4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net><8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi><4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi><19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> Message-ID: <7C0189D0CDB51646B7814F4377D356F504C1C35E@ADXV4.win.desy.de> Hi there, as I recall you are using OL2010 in an enterprise environment? In many cases home directories etc. are residing then on network shares. And that?s where .pst files and .ost files most probably are being written, too. When profiles are being configured writing incoming mails to .pst files (so you have a "local" copy), you will run in a situation, MS does not support/recommend, as this slows down (yes, confirmed) the client and may have negative side effects to others on that share: http://support.microsoft.com/kb/297019/en-us Cheers Dirk -- Dirk Jahnke-Zumbusch Deutsches Elektronen-Synchrotron DESY IT Information Fabrics Member of the Helmholtz Association D-22603 Hamburg Notkestrasse 85 / 22607 Hamburg From kayasaman at gmail.com Tue Jul 3 10:41:32 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 08:41:32 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are thereany tweaks? In-Reply-To: <7C0189D0CDB51646B7814F4377D356F504C1C35E@ADXV4.win.desy.de> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <7C0189D0CDB51646B7814F4377D356F504C1C35E@ADXV4.win.desy.de> Message-ID: On Tue, Jul 3, 2012 at 8:26 AM, Jahnke-Zumbusch, Dirk wrote: > Hi there, > > as I recall you are using OL2010 in an enterprise environment? In many > cases home directories etc. are residing then on network shares. And > that?s where .pst files and .ost files most probably are being written, > too. When profiles are being configured writing incoming mails to .pst > files (so you have a "local" copy), you will run in a situation, MS does > not support/recommend, as this slows down (yes, confirmed) the client > and may have negative side effects to others on that share: > > http://support.microsoft.com/kb/297019/en-us > > > Cheers > Dirk > > -- > Dirk Jahnke-Zumbusch Deutsches Elektronen-Synchrotron DESY > IT Information Fabrics Member of the Helmholtz Association > D-22603 Hamburg Notkestrasse 85 / 22607 Hamburg > Yeah the disk is relatively empty: # df -h Filesystem Size Used Avail Capacity Mounted on /dev/da0s1a 7.7G 5.2G 1.9G 74% / devfs 1.0K 1.0K 0B 100% /dev ZPOOL_1 9.8G 1.1G 8.7G 11% /mail The PST's seem to be stored on local hard disk too. It seems that there was an issue with Send/Recieve settings that my manager has made me aware of, basically turn off the Auto Send/Recieve and it seems to marginally improve performance?? Well thanks to all and I guess we'll keep testing ;-) Dovecot rocks, it's just a pitty that there is so much poorly programmed corporate software out there! Regards, Kaya From pw at wk-serv.de Tue Jul 3 12:28:55 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Tue, 03 Jul 2012 11:28:55 +0200 Subject: [Dovecot] error: director left/right disconnected Message-ID: <4FF2BB57.9070200@wk-serv.de> Hello, yesterday I set up a second director and even it works fine, some errors occur in the logfile on the new director: director: Error: Director 172.17.1.3:9090/left disconnected director: Error: Director 172.17.1.3:9090/right disconnected 172.17.1.3 belongs to the first director which has no errors. Where is the problem? Regards Patrick # 2.1.7: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_debug = yes auth_verbose = yes director_mail_servers = 172.17.1.1 director_servers = 172.17.1.3 172.17.1.4 lmtp_proxy = yes log_path = /var/log/dovecot.log managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = proxy=y nopassword=y starttls=any-cert driver = static } protocols = imap pop3 lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { address = 172.17.1.4 port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service imap-login { executable = imap-login director } service lmtp { inet_listener lmtp { address = 172.17.1.4 port = 24 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 4190 } } service pop3-login { executable = pop3-login director } ssl_cert = References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> Message-ID: <4FF2CB6A.8070106@Media-Brokers.com> On 2012-07-03 3:12 AM, Kaya Saman wrote: > However this is a clean server with plenty of space left on the pool > allocated for mail and it's additionally using ZFS too. What OS? ZFS implementation/version? How is mail stored (maildir? mbox?) While I don't think this is your problem, just fyi, my understanding is that it is fairly easy to implement ZFS wrong (which would cause serious performance problems), and that the only decent ZFS implementation is Suns (ie, what ships with Nexenta), or the latest FreeBSDs... Also, my understanding is that ZFS isn't the snappiest of filesystems even when properly configured (you trade performance for data integrity). Personally, I'd recommend trying this on a traditional FS (XFS or Reiserfs for maildir) and see if that changes things. On 2012-07-03 3:12 AM, Kaya Saman wrote: > The point is that I am monitoring using nload as well as other things > and the maximum bandwidth being got with Outlook is a few Mbps burst, > average 50kbps; while with T-Bird I get way over 130Mbps? Congrats - there's your problem... now you need to find out *why* this is so slow... most likely a tcp dump analysis of a session is the only way - I think there are people here who could help you analyze one (but not me, sorry)... On 2012-07-03 3:41 AM, Kaya Saman wrote: > The PST's seem to be stored on local hard disk too. 'Seem' to be? You need to make sure, because if they aren't that could definitely cause, or at least contribute to this kind of problem. -- Best regards, Charles From dovecot at beardz.net Tue Jul 3 13:44:11 2012 From: dovecot at beardz.net (Jase Thew) Date: Tue, 03 Jul 2012 11:44:11 +0100 Subject: [Dovecot] Problem with 'doveadm mailbox status -t' reporting cumulative vsizes after upgrading from v2.0.16 to v2.1.7 In-Reply-To: <1340789131.25551.48.camel@innu> References: <4FDF66E1.5050009@beardz.net> <1340789131.25551.48.camel@innu> Message-ID: <4FF2CCFB.3070200@beardz.net> On 27/06/2012 10:25, Timo Sirainen wrote: > On Mon, 2012-06-18 at 18:35 +0100, Jase Thew wrote: >> The reporting script at its core calls : >> >> doveadm -f flow mailbox status -A -t 'messages vsize' '*' >> >> It appears that Dovecot 2.1.7 is not resetting the vsize after collating >> the sum total of mailboxes sizes for each user, so that vsize just >> constantly increases as it iterates over each user. > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/d8d587bd5a29 > Thank you. Regards, Jase Thew. From kayasaman at gmail.com Tue Jul 3 13:51:34 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 11:51:34 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF2CB6A.8070106@Media-Brokers.com> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> Message-ID: On Tue, Jul 3, 2012 at 11:37 AM, Charles Marcus wrote: > On 2012-07-03 3:12 AM, Kaya Saman wrote: >> >> However this is a clean server with plenty of space left on the pool >> allocated for mail and it's additionally using ZFS too. > > > What OS? ZFS implementation/version? How is mail stored (maildir? mbox?) > > While I don't think this is your problem, just fyi, my understanding is that > it is fairly easy to implement ZFS wrong (which would cause serious > performance problems), and that the only decent ZFS implementation is Suns > (ie, what ships with Nexenta), or the latest FreeBSDs... > > Also, my understanding is that ZFS isn't the snappiest of filesystems even > when properly configured (you trade performance for data integrity). > > Personally, I'd recommend trying this on a traditional FS (XFS or Reiserfs > for maildir) and see if that changes things. FreeBSD 8.2 x64 using Maildir. ZFS is perfect no worries with that!!! Additionally the system is on a VMware cluster which is also fine - have checked all as diagnostics. The usage here is minimal, and since I also use ZFS at home too with quite a larger file system then at work (I know I know) and really hammer the heck out of it there is no issue. > > > On 2012-07-03 3:12 AM, Kaya Saman wrote: >> The point is that I am monitoring using nload as well as other things >> and the maximum bandwidth being got with Outlook is a few Mbps burst, >> average 50kbps; while with T-Bird I get way over 130Mbps? > > Congrats - there's your problem... now you need to find out *why* this is so > slow... most likely a tcp dump analysis of a session is the only way - I > think there are people here who could help you analyze one (but not me, > sorry)... > Yeah, it seems to be M$ implementation of IMAP. I don't think that there's anything anyone can do.... Outlook seems to wait after each transmission (found using Wireshark). > > On 2012-07-03 3:41 AM, Kaya Saman wrote: >> The PST's seem to be stored on local hard disk too. > > 'Seem' to be? You need to make sure, because if they aren't that could > definitely cause, or at least contribute to this kind of problem. > It is definitely stored locally! > -- > > Best regards, > > Charles Regards, Kaya From role.Dovecot-Readers at JLAssocs.com Tue Jul 3 13:57:02 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Tue, 3 Jul 2012 11:57:02 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> Message-ID: <9FC909EF-1A35-4B61-BBAF-AB04BE55DFAA@JLAssocs.com> On 3 Jul 2012, at 11:51, Kaya Saman wrote: > Yeah, it seems to be M$ implementation of IMAP. I don't think that > there's anything anyone can do.... Outlook seems to wait after each > transmission (found using Wireshark). Is the client syncing more than it has to? I mean, putting aside the delays in waiting on each transmission, is it generating traffic that shouldn't even be required? Still might not present a solution (short of unsubscribing much data) but I'm mindful of the oddity that disabling automatic send/receive makes such a difference. James. From kayasaman at gmail.com Tue Jul 3 14:00:41 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 12:00:41 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <9FC909EF-1A35-4B61-BBAF-AB04BE55DFAA@JLAssocs.com> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <9FC909EF-1A35-4B61-BBAF-AB04BE55DFAA@JLAssocs.com> Message-ID: On Tue, Jul 3, 2012 at 11:57 AM, J E Lyon wrote: > On 3 Jul 2012, at 11:51, Kaya Saman wrote: > >> Yeah, it seems to be M$ implementation of IMAP. I don't think that >> there's anything anyone can do.... Outlook seems to wait after each >> transmission (found using Wireshark). > > Is the client syncing more than it has to? I mean, putting aside the delays in waiting on each transmission, is it generating traffic that shouldn't even be required? Still might not present a solution (short of unsubscribing much data) but I'm mindful of the oddity that disabling automatic send/receive makes such a difference. > > James. Actually disabling Send/recieve didn't do anything :-( I held a stopwatch to it for both pre and post and no difference. T-Bird transfers get 1 min. 35 secs for 1600 messages. Outlook gets 3 min. 6 secs for the same amount of messages. Regards, Kaya From robert at schetterer.org Tue Jul 3 14:25:03 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 03 Jul 2012 13:25:03 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <9FC909EF-1A35-4B61-BBAF-AB04BE55DFAA@JLAssocs.com> Message-ID: <4FF2D68F.8000105@schetterer.org> Am 03.07.2012 13:00, schrieb Kaya Saman: > On Tue, Jul 3, 2012 at 11:57 AM, J E Lyon > wrote: >> On 3 Jul 2012, at 11:51, Kaya Saman wrote: >> >>> Yeah, it seems to be M$ implementation of IMAP. I don't think that >>> there's anything anyone can do.... Outlook seems to wait after each >>> transmission (found using Wireshark). >> >> Is the client syncing more than it has to? I mean, putting aside the delays in waiting on each transmission, is it generating traffic that shouldn't even be required? Still might not present a solution (short of unsubscribing much data) but I'm mindful of the oddity that disabling automatic send/receive makes such a difference. >> >> James. > > Actually disabling Send/recieve didn't do anything :-( > > > I held a stopwatch to it for both pre and post and no difference. > > T-Bird transfers get 1 min. 35 secs for 1600 messages. > > Outlook gets 3 min. 6 secs for the same amount of messages. > > > Regards, > > Kaya > meanwhile some workaround for the client http://www.youtube.com/watch?v=-6wXTP1AIq8 -- Best Regards MfG Robert Schetterer From kayasaman at gmail.com Tue Jul 3 14:32:37 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 12:32:37 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> Message-ID: On Tue, Jul 3, 2012 at 11:51 AM, Kaya Saman wrote: > On Tue, Jul 3, 2012 at 11:37 AM, Charles Marcus > wrote: >> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>> >>> However this is a clean server with plenty of space left on the pool >>> allocated for mail and it's additionally using ZFS too. >> >> >> What OS? ZFS implementation/version? How is mail stored (maildir? mbox?) >> >> While I don't think this is your problem, just fyi, my understanding is that >> it is fairly easy to implement ZFS wrong (which would cause serious >> performance problems), and that the only decent ZFS implementation is Suns >> (ie, what ships with Nexenta), or the latest FreeBSDs... >> >> Also, my understanding is that ZFS isn't the snappiest of filesystems even >> when properly configured (you trade performance for data integrity). >> >> Personally, I'd recommend trying this on a traditional FS (XFS or Reiserfs >> for maildir) and see if that changes things. > > FreeBSD 8.2 x64 using Maildir. ZFS is perfect no worries with that!!! > Additionally the system is on a VMware cluster which is also fine - > have checked all as diagnostics. > > The usage here is minimal, and since I also use ZFS at home too with > quite a larger file system then at work (I know I know) and really > hammer the heck out of it there is no issue. > >> >> >> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>> The point is that I am monitoring using nload as well as other things >>> and the maximum bandwidth being got with Outlook is a few Mbps burst, >>> average 50kbps; while with T-Bird I get way over 130Mbps? >> >> Congrats - there's your problem... now you need to find out *why* this is so >> slow... most likely a tcp dump analysis of a session is the only way - I >> think there are people here who could help you analyze one (but not me, >> sorry)... >> > > Yeah, it seems to be M$ implementation of IMAP. I don't think that > there's anything anyone can do.... Outlook seems to wait after each > transmission (found using Wireshark). > >> >> On 2012-07-03 3:41 AM, Kaya Saman wrote: >>> The PST's seem to be stored on local hard disk too. >> >> 'Seem' to be? You need to make sure, because if they aren't that could >> definitely cause, or at least contribute to this kind of problem. >> > > It is definitely stored locally! > >> -- >> >> Best regards, >> >> Charles > > Regards, > > > Kaya Ok now probably related to this is that some folders are not able to copy?? While dragging one folder from Outlook PST to the Dovecot IMAP server in Outlook 2010, the transfer keeps bombing out? In the logs all I see are: : Error: stat(/mail/AD_Mail// errors. My user was actually testing by copying the Inbox with many subdirectories into the INBOX on dovecot. Is this another Outlook related quirk or is it something serverside which I need to change? Regards, Kaya From kayasaman at gmail.com Tue Jul 3 14:36:43 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 12:36:43 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF2D68F.8000105@schetterer.org> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <9FC909EF-1A35-4B61-BBAF-AB04BE55DFAA@JLAssocs.com> <4FF2D68F.8000105@schetterer.org> Message-ID: On Tue, Jul 3, 2012 at 12:25 PM, Robert Schetterer wrote: > Am 03.07.2012 13:00, schrieb Kaya Saman: >> On Tue, Jul 3, 2012 at 11:57 AM, J E Lyon >> wrote: >>> On 3 Jul 2012, at 11:51, Kaya Saman wrote: >>> >>>> Yeah, it seems to be M$ implementation of IMAP. I don't think that >>>> there's anything anyone can do.... Outlook seems to wait after each >>>> transmission (found using Wireshark). >>> >>> Is the client syncing more than it has to? I mean, putting aside the delays in waiting on each transmission, is it generating traffic that shouldn't even be required? Still might not present a solution (short of unsubscribing much data) but I'm mindful of the oddity that disabling automatic send/receive makes such a difference. >>> >>> James. >> >> Actually disabling Send/recieve didn't do anything :-( >> >> >> I held a stopwatch to it for both pre and post and no difference. >> >> T-Bird transfers get 1 min. 35 secs for 1600 messages. >> >> Outlook gets 3 min. 6 secs for the same amount of messages. >> >> >> Regards, >> >> Kaya >> > > meanwhile some workaround for the client > > http://www.youtube.com/watch?v=-6wXTP1AIq8 > > -- > Best Regards > MfG Robert Schetterer > > Thanks! I tried disabling this for **all** accounts but found no perfomance benefit. Regards, Kaya From robert at schetterer.org Tue Jul 3 14:36:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 03 Jul 2012 13:36:59 +0200 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> Message-ID: <4FF2D95B.70806@schetterer.org> Am 03.07.2012 13:32, schrieb Kaya Saman: > On Tue, Jul 3, 2012 at 11:51 AM, Kaya Saman wrote: >> On Tue, Jul 3, 2012 at 11:37 AM, Charles Marcus >> wrote: >>> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>>> >>>> However this is a clean server with plenty of space left on the pool >>>> allocated for mail and it's additionally using ZFS too. >>> >>> >>> What OS? ZFS implementation/version? How is mail stored (maildir? mbox?) >>> >>> While I don't think this is your problem, just fyi, my understanding is that >>> it is fairly easy to implement ZFS wrong (which would cause serious >>> performance problems), and that the only decent ZFS implementation is Suns >>> (ie, what ships with Nexenta), or the latest FreeBSDs... >>> >>> Also, my understanding is that ZFS isn't the snappiest of filesystems even >>> when properly configured (you trade performance for data integrity). >>> >>> Personally, I'd recommend trying this on a traditional FS (XFS or Reiserfs >>> for maildir) and see if that changes things. >> >> FreeBSD 8.2 x64 using Maildir. ZFS is perfect no worries with that!!! >> Additionally the system is on a VMware cluster which is also fine - >> have checked all as diagnostics. >> >> The usage here is minimal, and since I also use ZFS at home too with >> quite a larger file system then at work (I know I know) and really >> hammer the heck out of it there is no issue. >> >>> >>> >>> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>>> The point is that I am monitoring using nload as well as other things >>>> and the maximum bandwidth being got with Outlook is a few Mbps burst, >>>> average 50kbps; while with T-Bird I get way over 130Mbps? >>> >>> Congrats - there's your problem... now you need to find out *why* this is so >>> slow... most likely a tcp dump analysis of a session is the only way - I >>> think there are people here who could help you analyze one (but not me, >>> sorry)... >>> >> >> Yeah, it seems to be M$ implementation of IMAP. I don't think that >> there's anything anyone can do.... Outlook seems to wait after each >> transmission (found using Wireshark). >> >>> >>> On 2012-07-03 3:41 AM, Kaya Saman wrote: >>>> The PST's seem to be stored on local hard disk too. >>> >>> 'Seem' to be? You need to make sure, because if they aren't that could >>> definitely cause, or at least contribute to this kind of problem. >>> >> >> It is definitely stored locally! >> >>> -- >>> >>> Best regards, >>> >>> Charles >> >> Regards, >> >> >> Kaya > > > Ok now probably related to this is that some folders are not able to copy?? > > While dragging one folder from Outlook PST to the Dovecot IMAP server > in Outlook 2010, the transfer keeps bombing out? > > In the logs all I see are: > > : Error: stat(/mail/AD_Mail// > > errors. > > My user was actually testing by copying the Inbox with many > subdirectories into the INBOX on dovecot. check pst file is local , check if copy from local over imap with subdir in general possible with outlook check no virus scanner proxies are involved > > > Is this another Outlook related quirk or is it something serverside > which I need to change? > > > Regards, > > > Kaya > -- Best Regards MfG Robert Schetterer From kayasaman at gmail.com Tue Jul 3 14:47:42 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 12:47:42 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <4FF2D95B.70806@schetterer.org> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <4FF2D95B.70806@schetterer.org> Message-ID: On Tue, Jul 3, 2012 at 12:36 PM, Robert Schetterer wrote: > Am 03.07.2012 13:32, schrieb Kaya Saman: >> On Tue, Jul 3, 2012 at 11:51 AM, Kaya Saman wrote: >>> On Tue, Jul 3, 2012 at 11:37 AM, Charles Marcus >>> wrote: >>>> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>>>> >>>>> However this is a clean server with plenty of space left on the pool >>>>> allocated for mail and it's additionally using ZFS too. >>>> >>>> >>>> What OS? ZFS implementation/version? How is mail stored (maildir? mbox?) >>>> >>>> While I don't think this is your problem, just fyi, my understanding is that >>>> it is fairly easy to implement ZFS wrong (which would cause serious >>>> performance problems), and that the only decent ZFS implementation is Suns >>>> (ie, what ships with Nexenta), or the latest FreeBSDs... >>>> >>>> Also, my understanding is that ZFS isn't the snappiest of filesystems even >>>> when properly configured (you trade performance for data integrity). >>>> >>>> Personally, I'd recommend trying this on a traditional FS (XFS or Reiserfs >>>> for maildir) and see if that changes things. >>> >>> FreeBSD 8.2 x64 using Maildir. ZFS is perfect no worries with that!!! >>> Additionally the system is on a VMware cluster which is also fine - >>> have checked all as diagnostics. >>> >>> The usage here is minimal, and since I also use ZFS at home too with >>> quite a larger file system then at work (I know I know) and really >>> hammer the heck out of it there is no issue. >>> >>>> >>>> >>>> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>>>> The point is that I am monitoring using nload as well as other things >>>>> and the maximum bandwidth being got with Outlook is a few Mbps burst, >>>>> average 50kbps; while with T-Bird I get way over 130Mbps? >>>> >>>> Congrats - there's your problem... now you need to find out *why* this is so >>>> slow... most likely a tcp dump analysis of a session is the only way - I >>>> think there are people here who could help you analyze one (but not me, >>>> sorry)... >>>> >>> >>> Yeah, it seems to be M$ implementation of IMAP. I don't think that >>> there's anything anyone can do.... Outlook seems to wait after each >>> transmission (found using Wireshark). >>> >>>> >>>> On 2012-07-03 3:41 AM, Kaya Saman wrote: >>>>> The PST's seem to be stored on local hard disk too. >>>> >>>> 'Seem' to be? You need to make sure, because if they aren't that could >>>> definitely cause, or at least contribute to this kind of problem. >>>> >>> >>> It is definitely stored locally! >>> >>>> -- >>>> >>>> Best regards, >>>> >>>> Charles >>> >>> Regards, >>> >>> >>> Kaya >> >> >> Ok now probably related to this is that some folders are not able to copy?? >> >> While dragging one folder from Outlook PST to the Dovecot IMAP server >> in Outlook 2010, the transfer keeps bombing out? >> >> In the logs all I see are: >> >> : Error: stat(/mail/AD_Mail// >> >> errors. >> >> My user was actually testing by copying the Inbox with many >> subdirectories into the INBOX on dovecot. > > check pst file is local , check if copy from local over imap with subdir > in general > possible with outlook > check no virus scanner proxies are involved > >> >> >> Is this another Outlook related quirk or is it something serverside >> which I need to change? >> >> >> Regards, >> >> >> Kaya >> > > > -- > Best Regards > MfG Robert Schetterer > > I attempted this myself as a check or test and Outlook claimed "Unable to open Deleted Items"? PST is local, subdirs are supported, no virus scanner or proxy in the way. Regards, Kaya From kayasaman at gmail.com Tue Jul 3 14:50:12 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 12:50:12 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <4FF2D95B.70806@schetterer.org> Message-ID: On Tue, Jul 3, 2012 at 12:47 PM, Kaya Saman wrote: > On Tue, Jul 3, 2012 at 12:36 PM, Robert Schetterer > wrote: >> Am 03.07.2012 13:32, schrieb Kaya Saman: >>> On Tue, Jul 3, 2012 at 11:51 AM, Kaya Saman wrote: >>>> On Tue, Jul 3, 2012 at 11:37 AM, Charles Marcus >>>> wrote: >>>>> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>>>>> >>>>>> However this is a clean server with plenty of space left on the pool >>>>>> allocated for mail and it's additionally using ZFS too. >>>>> >>>>> >>>>> What OS? ZFS implementation/version? How is mail stored (maildir? mbox?) >>>>> >>>>> While I don't think this is your problem, just fyi, my understanding is that >>>>> it is fairly easy to implement ZFS wrong (which would cause serious >>>>> performance problems), and that the only decent ZFS implementation is Suns >>>>> (ie, what ships with Nexenta), or the latest FreeBSDs... >>>>> >>>>> Also, my understanding is that ZFS isn't the snappiest of filesystems even >>>>> when properly configured (you trade performance for data integrity). >>>>> >>>>> Personally, I'd recommend trying this on a traditional FS (XFS or Reiserfs >>>>> for maildir) and see if that changes things. >>>> >>>> FreeBSD 8.2 x64 using Maildir. ZFS is perfect no worries with that!!! >>>> Additionally the system is on a VMware cluster which is also fine - >>>> have checked all as diagnostics. >>>> >>>> The usage here is minimal, and since I also use ZFS at home too with >>>> quite a larger file system then at work (I know I know) and really >>>> hammer the heck out of it there is no issue. >>>> >>>>> >>>>> >>>>> On 2012-07-03 3:12 AM, Kaya Saman wrote: >>>>>> The point is that I am monitoring using nload as well as other things >>>>>> and the maximum bandwidth being got with Outlook is a few Mbps burst, >>>>>> average 50kbps; while with T-Bird I get way over 130Mbps? >>>>> >>>>> Congrats - there's your problem... now you need to find out *why* this is so >>>>> slow... most likely a tcp dump analysis of a session is the only way - I >>>>> think there are people here who could help you analyze one (but not me, >>>>> sorry)... >>>>> >>>> >>>> Yeah, it seems to be M$ implementation of IMAP. I don't think that >>>> there's anything anyone can do.... Outlook seems to wait after each >>>> transmission (found using Wireshark). >>>> >>>>> >>>>> On 2012-07-03 3:41 AM, Kaya Saman wrote: >>>>>> The PST's seem to be stored on local hard disk too. >>>>> >>>>> 'Seem' to be? You need to make sure, because if they aren't that could >>>>> definitely cause, or at least contribute to this kind of problem. >>>>> >>>> >>>> It is definitely stored locally! >>>> >>>>> -- >>>>> >>>>> Best regards, >>>>> >>>>> Charles >>>> >>>> Regards, >>>> >>>> >>>> Kaya >>> >>> >>> Ok now probably related to this is that some folders are not able to copy?? >>> >>> While dragging one folder from Outlook PST to the Dovecot IMAP server >>> in Outlook 2010, the transfer keeps bombing out? >>> >>> In the logs all I see are: >>> >>> : Error: stat(/mail/AD_Mail// >>> >>> errors. >>> >>> My user was actually testing by copying the Inbox with many >>> subdirectories into the INBOX on dovecot. >> >> check pst file is local , check if copy from local over imap with subdir >> in general >> possible with outlook >> check no virus scanner proxies are involved >> >>> >>> >>> Is this another Outlook related quirk or is it something serverside >>> which I need to change? >>> >>> >>> Regards, >>> >>> >>> Kaya >>> >> >> >> -- >> Best Regards >> MfG Robert Schetterer >> >> > > I attempted this myself as a check or test and Outlook claimed "Unable > to open Deleted Items"? > > PST is local, subdirs are supported, no virus scanner or proxy in the way. > > Regards, > > Kaya Quick update: when transfering from Dovecot to Dovecot via Outlook I got a message popping up saying: "The move operation cannot be completed. It is possible that the destination server is unavailable or does not support subfolders" I think this is the standard error being seen.... Regards, Kaya From role.Dovecot-Readers at JLAssocs.com Tue Jul 3 15:03:35 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Tue, 3 Jul 2012 13:03:35 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> Message-ID: <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> On 3 Jul 2012, at 12:32, Kaya Saman wrote: > Ok now probably related to this is that some folders are not able to copy?? > > While dragging one folder from Outlook PST to the Dovecot IMAP server > in Outlook 2010, the transfer keeps bombing out? > > In the logs all I see are: > > : Error: stat(/mail/AD_Mail// > > errors. > > My user was actually testing by copying the Inbox with many > subdirectories into the INBOX on dovecot. > > > Is this another Outlook related quirk or is it something serverside > which I need to change? That's not something as simple as permissions on the server end, is it? You mentioned the INBOX... If you're talking Maildir then that means .../cur/ and not the "INBOX" directory itself of course, though I suspect it needs to exist. I'm trying to remember back when I had something like that happen and found the (unused) "INBOX" does need to exist. Something like that. J. From kayasaman at gmail.com Tue Jul 3 15:11:39 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 13:11:39 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> Message-ID: On Tue, Jul 3, 2012 at 1:03 PM, J E Lyon wrote: > On 3 Jul 2012, at 12:32, Kaya Saman wrote: > >> Ok now probably related to this is that some folders are not able to copy?? >> >> While dragging one folder from Outlook PST to the Dovecot IMAP server >> in Outlook 2010, the transfer keeps bombing out? >> >> In the logs all I see are: >> >> : Error: stat(/mail/AD_Mail// >> >> errors. >> >> My user was actually testing by copying the Inbox with many >> subdirectories into the INBOX on dovecot. >> >> >> Is this another Outlook related quirk or is it something serverside >> which I need to change? > > > That's not something as simple as permissions on the server end, is it? You mentioned the INBOX... If you're talking Maildir then that means .../cur/ and not the "INBOX" directory itself of course, though I suspect it needs to exist. I'm trying to remember back when I had something like that happen and found the (unused) "INBOX" does need to exist. Something like that. > > J. It is Maildir I am using, checked permissions - they're all ok. Yeah would be cur.... When connecting to this, do I need to put something like Inbox or INBOX as the mail root folder? I remember historically one needed to do that, however, with TBird one doens't need to any more though you still can..... Now to just get the Deleted Items working! Regards, Kaya From kayasaman at gmail.com Tue Jul 3 15:20:31 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 13:20:31 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> Message-ID: [...] > > > That's not something as simple as permissions on the server end, is it? I have my Maildir and parent folder permissions setup as: rwx-- mail_user:mail_user This should be ok shouldn't it or would I need to use rwxrwx- ?? By default it is created as stated at top of posting..... What confused me is that I was browsing around and people had different permissions settings on their Maildirs. Regards, Kaya From role.Dovecot-Readers at jlassocs.com Tue Jul 3 15:21:01 2012 From: role.Dovecot-Readers at jlassocs.com (J E Lyon) Date: Tue, 3 Jul 2012 13:21:01 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> Message-ID: On 3 Jul 2012, at 13:11, Kaya Saman wrote: > It is Maildir I am using, checked permissions - they're all ok. Yeah > would be cur.... When connecting to this, do I need to put something > like Inbox or INBOX as the mail root folder? > > I remember historically one needed to do that, however, with TBird one > doens't need to any more though you still can..... I'm sure the INBOX directory is created automatically for me, but I do have a script that sets up directories, permissions and I've taken care of SELinux issues in the past . . might be a red herring, was just mindful of the mention of INBOX -- does it work if you copy to another folder, or try to create a folder? > Now to just get the Deleted Items working! Ah well, I joined the list originally because of a query about the Deleted Items . . On Outlook pre-2010, that was a major headache, there's a plugin but it's one of only ?two not bundled by default with Dovecot (CentOS servers here, so using the default RPMs from there) . . I thought it was supposed to be better-behaved in 2010 thought . . . . J. From role.Dovecot-Readers at jlassocs.com Tue Jul 3 15:24:11 2012 From: role.Dovecot-Readers at jlassocs.com (J E Lyon) Date: Tue, 3 Jul 2012 13:24:11 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> Message-ID: <062ACC29-8B94-44C9-B988-6E979E6261EC@jlassocs.com> On 3 Jul 2012, at 13:20, Kaya Saman wrote: > [...] >> >> >> That's not something as simple as permissions on the server end, is it? > > > I have my Maildir and parent folder permissions setup as: > > rwx-- mail_user:mail_user I don't know what is strictly necessary, but I actually use rwxrws--- for my Maildir hierarchy, note the "s", don't think it's so important in more recent Dovecots but I set up my design originally on a Dovecot 0.99 and tweaked only slightly when moving to 1.x J. From kayasaman at gmail.com Tue Jul 3 15:32:27 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Tue, 3 Jul 2012 13:32:27 +0100 Subject: [Dovecot] Outlook 2010 very slow when using IMAP - are there any tweaks? In-Reply-To: References: <4FF1B2C6.1090404@svrinformatica.it> <4FF1B805.10603@coochey.net> <4FF1B8DA.3060200@coochey.net> <4FF1BCC9.9000301@coochey.net> <4FF1BEC0.4020007@coochey.net> <8AE20E01-9EE9-4AF4-B07A-6481D01869DE@iki.fi> <4137E77D-932E-4A07-93C4-E7EB16D07D4D@iki.fi> <19DC09FD-848C-4C92-AEE9-B7E9FC46317C@JLAssocs.com> <4FF2CB6A.8070106@Media-Brokers.com> <7B15AA13-9696-4F18-88CC-6E6640F66176@JLAssocs.com> Message-ID: On Tue, Jul 3, 2012 at 1:21 PM, J E Lyon wrote: > On 3 Jul 2012, at 13:11, Kaya Saman wrote: > >> It is Maildir I am using, checked permissions - they're all ok. Yeah >> would be cur.... When connecting to this, do I need to put something >> like Inbox or INBOX as the mail root folder? >> >> I remember historically one needed to do that, however, with TBird one >> doens't need to any more though you still can..... > > I'm sure the INBOX directory is created automatically for me, but I do have a script that sets up directories, permissions and I've taken care of SELinux issues in the past . . might be a red herring, was just mindful of the mention of INBOX -- does it work if you copy to another folder, or try to create a folder? Creating folders is fine and copying seems fine however, with quirks. > > >> Now to just get the Deleted Items working! > > Ah well, I joined the list originally because of a query about the Deleted Items . . On Outlook pre-2010, that was a major headache, there's a plugin but it's one of only ?two not bundled by default with Dovecot (CentOS servers here, so using the default RPMs from there) . . It's strange I just deleted some stuff in the "Deleted Items' folder through TBird which worked fine. However, previously nothing worked so I had to clear the whole dir out by diving into the server's file system and using rm -rf * > > I thought it was supposed to be better-behaved in 2010 thought . . . . > > > J. Regards, Kaya From voytek at sbt.net.au Tue Jul 3 16:09:29 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Tue, 3 Jul 2012 23:09:29 +1000 Subject: [Dovecot] + sub addressing setup In-Reply-To: <4FF1E525.6020606@localhost.localdomain.org> References: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> <4FF1E1A3.3010202@localhost.localdomain.org> <4FF1E525.6020606@localhost.localdomain.org> Message-ID: <4934f87e7cfe3ac357d3694ecdacb9d3.squirrel@sbt.net.au> On Tue, July 3, 2012 4:15 am, Pascal Volk wrote: > Oh, forget that tread. :) > I hope you are using Dovecot's lmtp, then set > lmtp_save_to_detail_mailbox = yes, in your 20-lmtp.conf. Otherwise pass use > the -m option with dovecot-lda (http://wiki2.dovecot.org/Tools/DovecotLDA) Pascal, thanks for your help OK, I wasn't, now, trying, using http://wiki2.dovecot.org/HowTo/PostfixDovecotLMTP but, I'm not getting very far: -------- The first step is to enable its stack via /etc/dovecot/dovecot.conf !include conf.d/*.conf protocols = imap lmtp -------- as soon as I past these two lines at the very end of /etc/dovecot.conf, and reload, I get like; Jul 3 22:58:16 dovecot: master: Warning: SIGHUP received - reloading configuration Jul 3 22:58:16 dovecot: master: Error: Error reading configuration: input is missing end-of-settings line -- Voytek From ckubu at so36.net Tue Jul 3 17:11:34 2012 From: ckubu at so36.net (ckubu) Date: Tue, 3 Jul 2012 16:11:34 +0200 Subject: [Dovecot] dict Panic after upgrade to 2.1.7 In-Reply-To: <7CEC4D79-FC7B-447E-9C1D-960B1D1606D2@iki.fi> References: <201206241221.16044.ckubu@so36.net> <7CEC4D79-FC7B-447E-9C1D-960B1D1606D2@iki.fi> Message-ID: <201207031611.34971.ckubu@so36.net> hallo, > > dovecot log entries: > > Jun 23 23:19:10 mx dovecot: dict: Panic: file driver-pgsql.c: line 84 > > (driver_pgsql_set_state): assertion failed: (state == SQL_DB_STATE_BUSY > > || db- > > > >> cur_result == NULL) > > This is clearly a bug, but I don't really see why it's happening. It would > be helpful if you could get a gdb backtrace: > http://dovecot.org/bugreport.html at all, i got 4 core dump files. Jul 3 15:16:57 mx dovecot: dict: Fatal: master: service(dict): child 11902 killed with signal 6 (core dumped) :qJul 3 15:22:59 mx dovecot: dict: Panic: file driver-pgsql.c: line 84 (driver_pgsql_set_state): assertion failed: (state == SQL_DB_STATE_BUSY || db->cur_result == NULL) Jul 3 15:22:59 mx dovecot: dict: Error: Raw backtrace: /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x4478a) [0x7f6ab8f1078a] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7f6ab8f107d6] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f6ab8ee75ef] -> dovecot/dict() [0x40a9a6] -> dovecot/dict() [0x40aa01] -> dovecot/dict() [0x40be43] -> dovecot/dict() [0x409474] -> dovecot/dict(sql_db_cache_deinit+0x20) [0x4089d0] -> dovecot/dict(main+0x169) [0x4059f9] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f6ab8960c8d] -> dovecot/dict() [0x404b59] Jul 3 15:22:59 mx dovecot: dict: Panic: file driver-pgsql.c: line 84 (driver_pgsql_set_state): assertion failed: (state == SQL_DB_STATE_BUSY || db->cur_result == NULL) Jul 3 15:22:59 mx dovecot: dict: Error: Raw backtrace: /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x4478a) [0x7f2d4160c78a] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7f2d4160c7d6] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(i_error+0) [0x7f2d415e35ef] -> dovecot/dict() [0x40a9a6] -> dovecot/dict() [0x40aa01] -> dovecot/dict() [0x40be43] -> dovecot/dict() [0x409474] -> dovecot/dict(sql_db_cache_deinit+0x20) [0x4089d0] -> dovecot/dict(main+0x169) [0x4059f9] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f2d4105cc8d] -> dovecot/dict() [0x404b59] Jul 3 15:22:59 mx dovecot: dict: Panic: file driver-pgsql.c: line 84 (driver_pgsql_set_state): assertion failed: (state == SQL_DB_STATE_BUSY || db->cur_result == NULL) Jul 3 15:22:59 mx dovecot: dict: Error: Raw backtrace: /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x4478a) [0x7fca2e05c78a] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7fca2e05c7d6] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fca2e0335ef] -> dovecot/dict() [0x40a9a6] -> dovecot/dict() [0x40aa01] -> dovecot/dict() [0x40be43] -> dovecot/dict() [0x409474] -> dovecot/dict(sql_db_cache_deinit+0x20) [0x4089d0] -> dovecot/dict(main+0x169) [0x4059f9] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7fca2daacc8d] -> dovecot/dict() [0x404b59] Jul 3 15:22:59 mx dovecot: dict: Fatal: master: service(dict): child 12033 killed with signal 6 (core dumped) Jul 3 15:22:59 mx dovecot: dict: Fatal: master: service(dict): child 12340 killed with signal 6 (core dumped) Jul 3 15:22:59 mx dovecot: dict: Fatal: master: service(dict): child 11847 killed with signal 6 (core dumped) i will give you the gdb backtrace from the first dump: gdb /usr/local/dovecot-2.1.7/libexec/dovecot/dict core.dict.11902 GNU gdb (GDB) 7.0.1-debian Copyright (C) 2009 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: ... Reading symbols from /usr/local/dovecot-2.1.7/libexec/dovecot/dict...done. Reading symbols from /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0...done. Loaded symbols for /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0 Reading symbols from /usr/lib/libpq.so.5...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libpq.so.5 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /usr/lib/libssl.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libssl.so.0.9.8 Reading symbols from /usr/lib/libcrypto.so.0.9.8...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libcrypto.so.0.9.8 Reading symbols from /usr/lib/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libkrb5.so.3 Reading symbols from /lib/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libcom_err.so.2 Reading symbols from /usr/lib/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgssapi_krb5.so.2 Reading symbols from /lib/libcrypt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libcrypt.so.1 Reading symbols from /usr/lib/libldap_r-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libldap_r-2.4.so.2 Reading symbols from /lib/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib/libpthread.so.0 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libz.so.1 Reading symbols from /usr/lib/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libk5crypto.so.3 Reading symbols from /usr/lib/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libkrb5support.so.0 Reading symbols from /lib/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libkeyutils.so.1 Reading symbols from /lib/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libresolv.so.2 Reading symbols from /usr/lib/liblber-2.4.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/liblber-2.4.so.2 Reading symbols from /usr/lib/libsasl2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libsasl2.so.2 Reading symbols from /usr/lib/libgnutls.so.26...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgnutls.so.26 Reading symbols from /usr/lib/libtasn1.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libtasn1.so.3 Reading symbols from /usr/lib/libgcrypt.so.11...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgcrypt.so.11 Reading symbols from /usr/lib/libgpg-error.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libgpg-error.so.0 Reading symbols from /lib/libgcc_s.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/libgcc_s.so.1 Core was generated by `dovecot/dict'. Program terminated with signal 6, Aborted. #0 0x00007ffc1f3da1b5 in raise () from /lib/libc.so.6 (gdb) bt full #0 0x00007ffc1f3da1b5 in raise () from /lib/libc.so.6 No symbol table info available. #1 0x00007ffc1f3dcfc0 in abort () from /lib/libc.so.6 No symbol table info available. #2 0x00007ffc1f97679d in default_fatal_finish (type=, status=0) at failures.c:191 backtrace = 0x1b10110 "/usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x4478a) [0x7ffc1f97678a] -> /usr/local/dovecot-2.1.7/lib/dovecot/libdovecot.so.0(+0x447d6) [0x7ffc1f9767d6] -> /usr/local/dovecot-2.1.7/lib/dovec"... #3 0x00007ffc1f9767d6 in i_internal_fatal_handler (ctx=0x7fff25ea55c0, format=, args=) at failures.c:649 status = 0 #4 0x00007ffc1f94d5ef in i_panic (format=0x2e7e
) at failures.c:263 ctx = {type = LOG_TYPE_PANIC, exit_status = 0, timestamp = 0x0} args = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7fff25ea5690, reg_save_area = 0x7fff25ea55d0}} #5 0x000000000040a9a6 in driver_pgsql_set_state (db=0x1b22280, state=) at driver-pgsql.c:84 __FUNCTION__ = "driver_pgsql_set_state" #6 0x000000000040aa01 in driver_pgsql_close (db=0x1b22280) at driver-pgsql.c:116 No locals. #7 0x000000000040be43 in driver_pgsql_deinit_v (_db=0x1b22280) at driver-pgsql.c:263 No locals. #8 0x0000000000409474 in driver_sqlpool_deinit (_db=0x1b21ee0) at driver-sqlpool.c:504 conn_end = 0x1b221a0 host = conn = 0x1b22180 __FUNCTION__ = "driver_sqlpool_deinit" #9 0x00000000004089d0 in sql_db_cache_deinit (_cache=) at sql-db-cache.c:142 cache = 0x1b1e460 #10 0x00000000004059f9 in main_deinit (argc=1, argv=0x1b183a0) at main.c:72 No locals. #11 main (argc=1, argv=0x1b183a0) at main.c:104 set_roots = {0x40c640, 0x0} error = 0x0 (gdb) quit Thank you for having a look. bw Christoph -- e: ckubu at so36.net From petros.fraser at gmail.com Tue Jul 3 23:07:14 2012 From: petros.fraser at gmail.com (Peter Fraser) Date: Tue, 3 Jul 2012 15:07:14 -0500 Subject: [Dovecot] Migration Question Message-ID: Hi all I got a shiny new server and I want to install the latest and greatest of everything on it. The old server has dovecot-1.2.17 working with postfix-2.8.4,1, dovecot quotas, sieve etc. My question is: Would it work of I installed dovecot 2 on this server along with everything I want and just copy the /home/virtualmail directory where all the user mail directories are? I use the Maildir format by the way. From wojtek at wojtek.tensor.gdynia.pl Tue Jul 3 23:34:05 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 3 Jul 2012 22:34:05 +0200 (CEST) Subject: [Dovecot] Migration Question In-Reply-To: References: Message-ID: > I got a shiny new server and I want to install the latest and greatest of latest may not necessarily be greatest. > everything on it. > The old server has dovecot-1.2.17 working > with postfix-2.8.4,1, dovecot quotas, sieve etc. > My question is: Would it work of I installed dovecot 2 on this server along > with everything I want and just copy the /home/virtualmail directory where no. config files have changed. After you will go through new configs everything else should work.. From tss at iki.fi Wed Jul 4 11:01:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 04 Jul 2012 11:01:12 +0300 Subject: [Dovecot] Removing specific entry in user/auth cache In-Reply-To: <42916718-6B7E-4632-8C61-AA8FE64D850E@iki.fi> References: <1340865829.25551.64.camel@innu> <42916718-6B7E-4632-8C61-AA8FE64D850E@iki.fi> Message-ID: <1341388872.2689.3.camel@innu> On Fri, 2012-06-29 at 05:01 +0300, Timo Sirainen wrote: > and for v2.1 a bit kludgy way: > > doveadm auth [] > doveadm auth cache flush [] Done: http://hg.dovecot.org/dovecot-2.1/rev/007bf0047ab0 http://hg.dovecot.org/dovecot-2.1/rev/1093c74f54af > so you couldn't test authentication against "cache" user, but that's probably not a problem. Actually you only can't test authentication against "cache" user with "flush" password. Even less likely to be a problem. From tim at tree-of-life.co.uk Wed Jul 4 19:17:55 2012 From: tim at tree-of-life.co.uk (Tim Dickson) Date: Wed, 04 Jul 2012 17:17:55 +0100 Subject: [Dovecot] files missed from install Message-ID: <4FF46CB3.9050901@tree-of-life.co.uk> version: using dovecot 2.1.8 or 2.0.12 bug: installation - some files missed details: dovecot-openssl.cnf and mkcert.sh are not copied from the src/doc folder to the installation doc folder on installation (make install) both are referred to in the documentation and are useful. is there any chance of updating the build scripts to include these files? thanks, Tim From user+dovecot at localhost.localdomain.org Wed Jul 4 20:33:35 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Wed, 04 Jul 2012 19:33:35 +0200 Subject: [Dovecot] + sub addressing setup In-Reply-To: <4934f87e7cfe3ac357d3694ecdacb9d3.squirrel@sbt.net.au> References: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> <4FF1E1A3.3010202@localhost.localdomain.org> <4FF1E525.6020606@localhost.localdomain.org> <4934f87e7cfe3ac357d3694ecdacb9d3.squirrel@sbt.net.au> Message-ID: <4FF47E6F.4030108@localhost.localdomain.org> On 07/03/2012 03:09 PM Voytek Eymont wrote: > ? > Jul 3 22:58:16 dovecot: master: Error: Error reading configuration: input > is missing end-of-settings line I guess there is somewhere a missing closing curly bracket in one of your configuration files. Regards, Pascal -- The trapper recommends today: deadbeef.1218619 at localdomain.org From wojtek at wojtek.tensor.gdynia.pl Wed Jul 4 21:33:11 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Wed, 4 Jul 2012 20:33:11 +0200 (CEST) Subject: [Dovecot] files missed from install In-Reply-To: <4FF46CB3.9050901@tree-of-life.co.uk> References: <4FF46CB3.9050901@tree-of-life.co.uk> Message-ID: > > is there any chance of updating the build scripts to include these files? > thanks, same here. i've copied it from dovecot 1 install From amateo at um.es Wed Jul 4 21:49:21 2012 From: amateo at um.es (Angel L. Mateo) Date: Wed, 04 Jul 2012 20:49:21 +0200 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations Message-ID: <4FF49031.60701@um.es> Hello, We are having performance problems trying to migrate our pop/imap servers to a new version. Our old servers are 4 debian lenny with 5GB of RAM running of XenServer VMs with kernel 2.6.32-4-amd64 and dovecot 1.1.16. New servers are 4 ubuntu 12.04 with dovecot 2.1.5 running on vmware vm with 6 cores and 16GB of RAM and kernel 3.2.0-24-generic. On both server we are using nfs 3 with same configuration (regardless of internal kernel differences, but we have no customized any of them, we are using vanilla kernels with default configurations). The problem we have is that new servers have performance problems. Even when have a small part of our total users (about 25%) directed to the new farm, performance is very poor, even useless. Looking for NFS problems, we have found a lot of differences in nfs operations. For example, this is the nfsstat of one of a new servers at this moment: myotis21:~# nfsstat Client rpc stats: calls retrans authrefrsh 414528349 885 37 Client nfs v3: null getattr setattr lookup access readlink 0 0% 95673837 23% 3961938 0% 89586364 21% 110097351 26% 2930961 0% read write create mkdir symlink mknod 20009850 4% 6065319 1% 3757720 0% 1557 0% 0 0% 0 0% remove rmdir rename link readdir readdirplus 6378134 1% 281 0% 2602358 0% 555097 0% 53126619 12% 15615402 3% fsstat fsinfo pathconf commit 113256 0% 26152 0% 0 0% 4026151 0% and this is the same on one of the new ones: amateo_adm at myotis31:~$ nfsstat Server rpc stats: calls badcalls badclnt badauth xdrcall 0 0 0 0 0 Client rpc stats: calls retrans authrefrsh 178040318 675 178040800 Client nfs v3: null getattr setattr lookup access readlink 0 0% 24350345 13% 5045924 2% 10939469 6% 30185146 16% 142865 0% read write create mkdir symlink mknod 8818016 4% 6058614 3% 2877653 1% 420 0% 0 0% 0 0% remove rmdir rename link readdir readdirplus 2842562 1% 69 0% 2961239 1% 634038 0% 0 0% 82921863 46% fsstat fsinfo pathconf commit 70861 0% 18754 0% 9377 0% 152702 0% Although nfs configuration is the same, there are a lot of differences on readdir vs readdirplus nfs operations. In fact, in the old one we have 12% readdir operations and 3% of readdirplus. And in the new one we have 46% of readdirplus and no readdir operations. Although readdirplus is supposed to be an optimization in nfs3, in situations when you have big directories and uses just a few entries of these directories it could be worse. So we think if this could be the problem (or one of them). Any idea of this difference? And if this difference could be significative? PS: I have attached doveconf -n of the new server. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 -------------- next part -------------- # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-24-generic x86_64 Ubuntu 12.04 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1000 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_nfs_storage = yes mail_privileged_group = mail mdbox_rotate_size = 20 M passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = .EXPUNGED/ .DELETED/ .DELETED/.EXPUNGED/ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = Hi all What is the best strategy to add another storage to an existing virtual mail system ? Move some domains to the new storage and create symlinks ? Switch to dovecot hashing ? But in this case what is the easy-east way to migrate ? Thanks for any suggestions or tips ! From role.Dovecot-Readers at JLAssocs.com Wed Jul 4 23:22:30 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Wed, 4 Jul 2012 21:22:30 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF4A12C.30704@gmail.com> References: <4FF4A12C.30704@gmail.com> Message-ID: On 4 Jul 2012, at 21:01, Adrian Minta wrote: > What is the best strategy to add another storage to an existing virtual mail system ? > Move some domains to the new storage and create symlinks ? > Switch to dovecot hashing ? But in this case what is the easy-east way to migrate ? > > Thanks for any suggestions or tips ! Are you using Volume Management (VLM) on the system, or do you have regular partitions mounted? Is there any RAID or other factors to consider . . in fact, a few details about your system might help :) ~ James. From adrian.minta at gmail.com Thu Jul 5 00:09:31 2012 From: adrian.minta at gmail.com (Adrian Minta) Date: Thu, 05 Jul 2012 00:09:31 +0300 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> Message-ID: <4FF4B10B.9070906@gmail.com> On 07/04/12 23:22, J E Lyon wrote: > On 4 Jul 2012, at 21:01, Adrian Minta wrote: > >> What is the best strategy to add another storage to an existing virtual mail system ? >> Move some domains to the new storage and create symlinks ? >> Switch to dovecot hashing ? But in this case what is the easy-east way to migrate ? >> >> Thanks for any suggestions or tips ! > Are you using Volume Management (VLM) on the system, or do you have regular partitions mounted? Is there any RAID or other factors to consider . . in fact, a few details about your system might help :) > > ~ James. No LVM and the RAID is done in the SAN appliance. My gut tells me that formatting a single huge 25TB partition is not the best way to go. That's why I consider having two different LUN's on two different SAN's mounted together as two directories. A fsck and a recovery will work faster. The only issue is distributing maildirs. across the two file systems. From role.Dovecot-Readers at JLAssocs.com Thu Jul 5 00:25:39 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Wed, 4 Jul 2012 22:25:39 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF4B10B.9070906@gmail.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> Message-ID: <4D695F9E-B502-4FA1-A1F3-3F93D394F8C0@JLAssocs.com> On 4 Jul 2012, at 22:09, Adrian Minta wrote: > On 07/04/12 23:22, J E Lyon wrote: >> On 4 Jul 2012, at 21:01, Adrian Minta wrote: >> >>> What is the best strategy to add another storage to an existing virtual mail system ? >>> Move some domains to the new storage and create symlinks ? >>> Switch to dovecot hashing ? But in this case what is the easy-east way to migrate ? >>> >>> Thanks for any suggestions or tips ! >> Are you using Volume Management (VLM) on the system, or do you have regular partitions mounted? Is there any RAID or other factors to consider . . in fact, a few details about your system might help :) >> >> ~ James. > > No LVM and the RAID is done in the SAN appliance. > My gut tells me that formatting a single huge 25TB partition is not the best way to go. That's why I consider having two different LUN's on two different SAN's mounted together as two directories. A fsck and a recovery will work faster. > The only issue is distributing maildirs. across the two file systems. LVM would help with all the above -- sounds like it's not an option though. I have seen recommendations for Maildir structures whereby the first letter of the user account is one of 26 (or however many) directories, and the accounts therefore fall into one of several different directories -- you can distribute the letters of the alphabet directories across the two filesystems . . Have you seen the configuration examples that use parameters to specify the Maildir locations based on the account name etc.? ~ James. From tss at iki.fi Thu Jul 5 00:55:46 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Jul 2012 00:55:46 +0300 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <4FF49031.60701@um.es> References: <4FF49031.60701@um.es> Message-ID: On 4.7.2012, at 21.49, Angel L. Mateo wrote: > Although nfs configuration is the same, there are a lot of differences on readdir vs readdirplus nfs operations. In fact, in the old one we have 12% readdir operations and 3% of readdirplus. And in the new one we have 46% of readdirplus and no readdir operations. I'm not entirely sure, but I think it's the kernel that decides if readdir or readdirplus is used and Dovecot can't affect that decision. (Unless maybe kernel does some heuristics.) > PS: I have attached doveconf -n of the new server. At least this reduces performance: mail_nfs_storage = yes Also maildir_very_dirty_syncs=yes improves performance by reducing readdirs. It's safe to use as long as only Dovecot is reading the Maildir. From daniel.parthey at informatik.tu-chemnitz.de Thu Jul 5 01:02:28 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 5 Jul 2012 00:02:28 +0200 Subject: [Dovecot] error: director left/right disconnected In-Reply-To: <4FF2BB57.9070200@wk-serv.de> References: <4FF2BB57.9070200@wk-serv.de> Message-ID: <20120704220228.GA8143@daniel.localdomain> Patrick Westenberg wrote: > Hello, > > yesterday I set up a second director and even it works fine, some > errors occur in the logfile on the new director: > > director: Error: Director 172.17.1.3:9090/left disconnected > director: Error: Director 172.17.1.3:9090/right disconnected > > 172.17.1.3 belongs to the first director which has no errors. > > Where is the problem? Are you using different instances for director and mailboxes, or do you run them on dedicated hosts? Can you telnet to the director port from one host to another? 172.17.1.3# telnet 172.17.1.4 9090 172.17.1.4# telnet 172.17.1.3 9090 Is the ring connected correctly now? What does "doveadm -c director.conf director ring status" show? How often did these "disconnected" log entries appear, only once or repeatedly? Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Thu Jul 5 01:19:58 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Thu, 5 Jul 2012 00:19:58 +0200 Subject: [Dovecot] Migration Question In-Reply-To: References: Message-ID: <20120704221957.GA8521@daniel.localdomain> Hi Peter, Peter Fraser wrote: > I got a shiny new server and I want to install the latest and greatest of > everything on it. The old server has dovecot-1.2.17 working > with postfix-2.8.4,1, dovecot quotas, sieve etc. > My question is: Would it work of I installed dovecot 2 on this server along > with everything I want and just copy the /home/virtualmail directory where > all the user mail directories are? I use the Maildir format by the way. Please read carefully the following wiki article: http://wiki2.dovecot.org/Upgrading/2.0 It describes how to convert your old configs using doveconf and which changes you should pay attention to. Regards Daniel -- https://plus.google.com/103021802792276734820 From stan at hardwarefreak.com Thu Jul 5 07:37:40 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Wed, 04 Jul 2012 23:37:40 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF4B10B.9070906@gmail.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> Message-ID: <4FF51A14.70102@hardwarefreak.com> On 7/4/2012 4:09 PM, Adrian Minta wrote: > On 07/04/12 23:22, J E Lyon wrote: >> On 4 Jul 2012, at 21:01, Adrian Minta wrote: >> >>> What is the best strategy to add another storage to an existing >>> virtual mail system ? >>> Move some domains to the new storage and create symlinks ? >>> Switch to dovecot hashing ? But in this case what is the easy-east >>> way to migrate ? >>> >>> Thanks for any suggestions or tips ! >> Are you using Volume Management (VLM) on the system, or do you have >> regular partitions mounted? Is there any RAID or other factors to >> consider . . in fact, a few details about your system might help :) >> >> ~ James. > > No LVM and the RAID is done in the SAN appliance. It absolutely kills me every time I see a mail server admin display almost total lack of knowledge of his/her storage back end, or the inability to describe it technically, in an email... > My gut tells me that formatting a single huge 25TB partition is not the > best way to go. What is this statement supposed to convey to us? It makes no sense. > That's why I consider having two different LUN's on two > different SAN's mounted together as two directories. And this doesn't make sense either. > A fsck and a > recovery will work faster. If you're having to run fsck on your filesystem on a regular basis, you've have larger problems than storage provisioning. What filesystem are you using? > The only issue is distributing maildirs. across the two file systems. Here's the type of information and level of detail you need to provide for any of us to intelligently assist you with this issue: My SAN array is model X from company Y. It has 12x 3TB drives configured as a RAID6 array. I have created a 5TB virtual drive and exported it as LUN #X. It houses the filesystem (extX/XFS/JFS) that contains the mail store, which is now getting full. I have 25TB of the 30TB net array space unallocated. How can I best use this space to increase storage space for Dovecot? To which I would answer: Unless your SAN array is an el cheap model, you should be able to simply increase the size of the current virtual drive by adding a portion of the 25TB of unallocated space. Say you expand the virtual drive by 10TB. The virtual drive is already exported as LUN #X, so the host to which the LUN is unmasked simply now sees /dev/sdX as being 15TB total size instead of 5TB. At this point, simply grow the filesystem across the additional 10TB of the "disk". This negates the need for multiple filesystems/namespaces and jumping through hoops to balance your maildir workload across two separate filesystems. This "on the fly" expansion capability is one of the biggest selling points of SAN technology. Surely your unit has such capability. -- Stan From amateo at um.es Thu Jul 5 08:44:36 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 05 Jul 2012 07:44:36 +0200 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: References: <4FF49031.60701@um.es> Message-ID: <4FF529C4.90800@um.es> El 04/07/12 23:55, Timo Sirainen escribi?: > Also maildir_very_dirty_syncs=yes improves performance by reducing readdirs. It's safe to use as long as only Dovecot is reading the Maildir. > Is it safe to use it although a user could have open simultaneous sessions in different servers? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tss at iki.fi Thu Jul 5 08:49:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Jul 2012 08:49:33 +0300 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <4FF529C4.90800@um.es> References: <4FF49031.60701@um.es> <4FF529C4.90800@um.es> Message-ID: On 5.7.2012, at 8.44, Angel L. Mateo wrote: > El 04/07/12 23:55, Timo Sirainen escribi?: > >> Also maildir_very_dirty_syncs=yes improves performance by reducing readdirs. It's safe to use as long as only Dovecot is reading the Maildir. >> > > Is it safe to use it although a user could have open simultaneous sessions in different servers? Each server has local indexes? Then it's not very optimal.. but it doesn't break anything badly, Dovecot just might not immediately notice when another server has done changes. From amateo at um.es Thu Jul 5 09:52:15 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 05 Jul 2012 08:52:15 +0200 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: References: <4FF49031.60701@um.es> <4FF529C4.90800@um.es> Message-ID: <4FF5399F.2090207@um.es> El 05/07/12 07:49, Timo Sirainen escribi?: > On 5.7.2012, at 8.44, Angel L. Mateo wrote: > >> El 04/07/12 23:55, Timo Sirainen escribi?: >> >>> Also maildir_very_dirty_syncs=yes improves performance by reducing readdirs. It's safe to use as long as only Dovecot is reading the Maildir. >>> >> >> Is it safe to use it although a user could have open simultaneous sessions in different servers? > > Each server has local indexes? Then it's not very optimal.. but it doesn't break anything badly, Dovecot just might not immediately notice when another server has done changes. > Yes, indexes are local. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From adrian.minta at gmail.com Thu Jul 5 10:44:19 2012 From: adrian.minta at gmail.com (Adrian M) Date: Thu, 5 Jul 2012 10:44:19 +0300 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF51A14.70102@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: On Thu, Jul 5, 2012 at 7:37 AM, Stan Hoeppner wrote: > On 7/4/2012 4:09 PM, Adrian Minta wrote: >> On 07/04/12 23:22, J E Lyon wrote: >>> On 4 Jul 2012, at 21:01, Adrian Minta wrote: >>> >>>> What is the best strategy to add another storage to an existing >>>> virtual mail system ? >>>> Move some domains to the new storage and create symlinks ? >>>> Switch to dovecot hashing ? But in this case what is the easy-east >>>> way to migrate ? >>>> >>>> Thanks for any suggestions or tips ! >>> Are you using Volume Management (VLM) on the system, or do you have >>> regular partitions mounted? Is there any RAID or other factors to >>> consider . . in fact, a few details about your system might help :) >>> >>> ~ James. >> >> No LVM and the RAID is done in the SAN appliance. > > It absolutely kills me every time I see a mail server admin display > almost total lack of knowledge of his/her storage back end, or the > inability to describe it technically, in an email... > >> My gut tells me that formatting a single huge 25TB partition is not the >> best way to go. > > What is this statement supposed to convey to us? It makes no sense. > >> That's why I consider having two different LUN's on two >> different SAN's mounted together as two directories. > > And this doesn't make sense either. > >> A fsck and a >> recovery will work faster. > > If you're having to run fsck on your filesystem on a regular basis, > you've have larger problems than storage provisioning. What filesystem > are you using? > >> The only issue is distributing maildirs. across the two file systems. > > Here's the type of information and level of detail you need to provide > for any of us to intelligently assist you with this issue: > > My SAN array is model X from company Y. It has 12x 3TB drives > configured as a RAID6 array. I have created a 5TB virtual drive and > exported it as LUN #X. It houses the filesystem (extX/XFS/JFS) that > contains the mail store, which is now getting full. I have 25TB of the > 30TB net array space unallocated. How can I best use this space to > increase storage space for Dovecot? > > To which I would answer: > > Unless your SAN array is an el cheap model, you should be able to simply > increase the size of the current virtual drive by adding a portion of > the 25TB of unallocated space. Say you expand the virtual drive by > 10TB. The virtual drive is already exported as LUN #X, so the host to > which the LUN is unmasked simply now sees /dev/sdX as being 15TB total > size instead of 5TB. At this point, simply grow the filesystem across > the additional 10TB of the "disk". This negates the need for multiple > filesystems/namespaces and jumping through hoops to balance your maildir > workload across two separate filesystems. > > This "on the fly" expansion capability is one of the biggest selling > points of SAN technology. Surely your unit has such capability. > > -- > Stan Hi Stan, I know how to add drives to the storage and how to grow the existing filesystem, but such big filesystems are somehow new to mainstream linux. Yes, I know some university out there already have pentabytes filesystems, but right now stable linux systems have trouble formatting ext4 partition over 16T. All this is telling me that is safer to have two or tree smaller filesystems than a big one. Dovecot has a nice feature for this "Directory hashing" http://wiki.dovecot.org/MailLocation/ What I don't know is a nice way to migrate from a single directory no hashing to more than one and hashing. From tss at iki.fi Thu Jul 5 10:59:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Jul 2012 10:59:04 +0300 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: <1D912C03-501A-42FC-8A9D-A6C72A7C32D8@iki.fi> On 5.7.2012, at 10.44, Adrian M wrote: > All this is telling me that is safer to have two or tree smaller > filesystems than a big one. Dovecot has a nice feature for this > "Directory hashing" http://wiki.dovecot.org/MailLocation/ > > What I don't know is a nice way to migrate from a single directory no > hashing to more than one and hashing. Alternative to hashing is to simply return a "mail" or "home" setting from userdb pointing to your new mountpoint. From stan at hardwarefreak.com Thu Jul 5 12:35:09 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 05 Jul 2012 04:35:09 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: <4FF55FCD.6070001@hardwarefreak.com> On 7/5/2012 2:44 AM, Adrian M wrote: > Hi Stan, > I know how to add drives to the storage and how to grow the existing > filesystem, but such big filesystems are somehow new to mainstream > linux. Yes, I know some university out there already have pentabytes > filesystems, but right now stable linux systems have trouble > formatting ext4 partition over 16T. > All this is telling me that is safer to have two or tree smaller > filesystems than a big one. Dovecot has a nice feature for this > "Directory hashing" http://wiki.dovecot.org/MailLocation/ At 16TB+ scale with maildir you should be using XFS on kernel 3.x, not EXT4. Your performance will be significantly better, as in 30% or much more. The typical XFS filesystem in the wild today is 50TB+ and there are hundreds of XFS filesystems well over 100TB deployed around the world. NASA has XFS filesystems of 380TB and 535TB, and also has multiple 1PB+ CXFS (cluster XFS) filesystems. 20TB is a tiny snack for XFS, 500TB is lunch, 1PB is a big supper. A single XFS can scale to 16 Exabytes, or 1 million terabytes, though the largest deployed so far that I'm aware of is NASA's 535TB XFS. It'll scale to anything you'll ever throw at it, and much more. > What I don't know is a nice way to migrate from a single directory no > hashing to more than one and hashing. It's a good time to migrate to XFS. -- Stan From kayasaman at gmail.com Thu Jul 5 12:45:00 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Thu, 5 Jul 2012 10:45:00 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF55FCD.6070001@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: On Thu, Jul 5, 2012 at 10:35 AM, Stan Hoeppner wrote: > On 7/5/2012 2:44 AM, Adrian M wrote: > >> Hi Stan, >> I know how to add drives to the storage and how to grow the existing >> filesystem, but such big filesystems are somehow new to mainstream >> linux. Yes, I know some university out there already have pentabytes >> filesystems, but right now stable linux systems have trouble >> formatting ext4 partition over 16T. >> All this is telling me that is safer to have two or tree smaller >> filesystems than a big one. Dovecot has a nice feature for this >> "Directory hashing" http://wiki.dovecot.org/MailLocation/ > > At 16TB+ scale with maildir you should be using XFS on kernel 3.x, not > EXT4. Your performance will be significantly better, as in 30% or much > more. The typical XFS filesystem in the wild today is 50TB+ and there > are hundreds of XFS filesystems well over 100TB deployed around the world. > > NASA has XFS filesystems of 380TB and 535TB, and also has multiple 1PB+ > CXFS (cluster XFS) filesystems. 20TB is a tiny snack for XFS, 500TB is > lunch, 1PB is a big supper. A single XFS can scale to 16 Exabytes, or 1 > million terabytes, though the largest deployed so far that I'm aware of > is NASA's 535TB XFS. It'll scale to anything you'll ever throw at it, > and much more. > >> What I don't know is a nice way to migrate from a single directory no >> hashing to more than one and hashing. > > It's a good time to migrate to XFS. > > -- > Stan Other good operating systems for large filesystems are FreeBSD with either UFS2 or ZFS, or Solaris with ZFS. But then one must think, do I really want to switch OS? As a personal preference I would go FreeBSD route as dovecot is easily available through 'Ports'. Even with miniscule stroage (20GB) for my new Dovecot IMAP server here at work FreeBSD 8.2 x64 running on VMware using ZFS for the 'extra' 20GB disk is performing pretty well. Both UFS2 and ZFS can be snapshotted although ZFS has some functionality which is really cool but one will need to know how to set it up properly. Hence Zeta Byte Filesystem :-) UFS2 tends to perform faster.... however. Regards, Kaya From role.Dovecot-Readers at JLAssocs.com Thu Jul 5 12:46:45 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Thu, 5 Jul 2012 10:46:45 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: On 5 Jul 2012, at 08:44, Adrian M wrote: > Hi Stan, > I know how to add drives to the storage and how to grow the existing > filesystem, but such big filesystems are somehow new to mainstream > linux. Yes, I know some university out there already have pentabytes > filesystems, but right now stable linux systems have trouble > formatting ext4 partition over 16T. > All this is telling me that is safer to have two or tree smaller > filesystems than a big one. Dovecot has a nice feature for this > "Directory hashing" http://wiki.dovecot.org/MailLocation/ > > What I don't know is a nice way to migrate from a single directory no > hashing to more than one and hashing. When I first saw you mention hashing, I misread it as some sort of hash-table approach to large directories that I wasn't aware of, or something . . And now I've read the Dovecot documentation, I see what you're talking about! Why are you bothered about using the hash, instead of just splitting on the first letter of the existing account name? Is it to more evenly randomise the distribution of accounts? The advantage of the username "first letter" approach, or returning hardcoded locations from userdb (as per Timo's suggestion), is that you can more readily move the directories manually during a very short period of downtime, I'd have thought? ~ James. From role.Dovecot-Readers at JLAssocs.com Thu Jul 5 12:48:39 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Thu, 5 Jul 2012 10:48:39 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: On 5 Jul 2012, at 10:45, Kaya Saman wrote: > But then one must think, do I really want to switch OS? I heard a rumour that switching OS is sometimes harder than adding a mountpoint :) J. From kayasaman at gmail.com Thu Jul 5 12:55:20 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Thu, 5 Jul 2012 10:55:20 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: On Thu, Jul 5, 2012 at 10:48 AM, J E Lyon wrote: > On 5 Jul 2012, at 10:45, Kaya Saman wrote: > >> But then one must think, do I really want to switch OS? > > I heard a rumour that switching OS is sometimes harder than adding a mountpoint :) > > J. It can be! That's why I'm not even thinking of migrating the mission critical stuff running on CentOS 5 to even CentOS 6 yet. Am however, trying to do all clean installs on FreeBSD where I **can ** get away with it. Ok this may sound incredibly sad so don't sue me for it, but for my OpenSource work at home I have switched over from 15+ Linux servers down to 1x FreeBSD system running Jails. The statement about switching OS's was only a suggestion though with a rear-door retreat open incase it failed :-) Regards, Kaya From role.Dovecot-Readers at jlassocs.com Thu Jul 5 13:01:32 2012 From: role.Dovecot-Readers at jlassocs.com (J E Lyon) Date: Thu, 5 Jul 2012 11:01:32 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: <43E401E6-DC6D-46B2-94D4-7A961C177ED6@jlassocs.com> On 5 Jul 2012, at 10:55, Kaya Saman wrote: > That's why I'm not even thinking of migrating the mission critical > stuff running on CentOS 5 to even CentOS 6 yet. I'm in an identical position there -- and in fact, I think it's time to get some virtualised hosting of CentOS 6 servers, once I decide on the choice of underlying solution . . feels like a minefield at this point in time . . I've been working with RedHat then RH & Fedora then CentOS since the 90s, but haven't any firsthand experience of installing & maintaining VMWare or anything like it. Exciting times :) > Ok this may sound incredibly sad so don't sue me for it, but for my > OpenSource work at home I have switched over from 15+ Linux servers > down to 1x FreeBSD system running Jails. Ah yes... see above :) J. From adrian.minta at gmail.com Thu Jul 5 13:10:31 2012 From: adrian.minta at gmail.com (Adrian M) Date: Thu, 5 Jul 2012 13:10:31 +0300 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF55FCD.6070001@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: On Thu, Jul 5, 2012 at 12:35 PM, Stan Hoeppner wrote: > On 7/5/2012 2:44 AM, Adrian M wrote: > >> Hi Stan, >> I know how to add drives to the storage and how to grow the existing >> filesystem, but such big filesystems are somehow new to mainstream >> linux. Yes, I know some university out there already have pentabytes >> filesystems, but right now stable linux systems have trouble >> formatting ext4 partition over 16T. >> All this is telling me that is safer to have two or tree smaller >> filesystems than a big one. Dovecot has a nice feature for this >> "Directory hashing" http://wiki.dovecot.org/MailLocation/ > > At 16TB+ scale with maildir you should be using XFS on kernel 3.x, not > EXT4. Your performance will be significantly better, as in 30% or much > more. The typical XFS filesystem in the wild today is 50TB+ and there > are hundreds of XFS filesystems well over 100TB deployed around the world. > > NASA has XFS filesystems of 380TB and 535TB, and also has multiple 1PB+ > CXFS (cluster XFS) filesystems. 20TB is a tiny snack for XFS, 500TB is > lunch, 1PB is a big supper. A single XFS can scale to 16 Exabytes, or 1 > million terabytes, though the largest deployed so far that I'm aware of > is NASA's 535TB XFS. It'll scale to anything you'll ever throw at it, > and much more. > >> What I don't know is a nice way to migrate from a single directory no >> hashing to more than one and hashing. > > It's a good time to migrate to XFS. > > -- > Stan Hi Stan, I already have xfs, and kernel 3.x. Switching from kernel 2.6.x to 3.x a couple months ago did indeed decrease the iops number. From adrian.minta at gmail.com Thu Jul 5 13:14:59 2012 From: adrian.minta at gmail.com (Adrian M) Date: Thu, 5 Jul 2012 13:14:59 +0300 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: On Thu, Jul 5, 2012 at 12:46 PM, J E Lyon wrote: > > When I first saw you mention hashing, I misread it as some sort of hash-table approach to large directories that I wasn't aware of, or something . . And now I've read the Dovecot documentation, I see what you're talking about! > > Why are you bothered about using the hash, instead of just splitting on the first letter of the existing account name? Is it to more evenly randomise the distribution of accounts? > > The advantage of the username "first letter" approach, or returning hardcoded locations from userdb (as per Timo's suggestion), is that you can more readily move the directories manually during a very short period of downtime, I'd have thought? > > > ~ James. Thank you, this is a valid suggestion, especially since it could be done directly with some SQL magic in dovecot config file. I will consider this option ! From kayasaman at gmail.com Thu Jul 5 13:15:47 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Thu, 5 Jul 2012 11:15:47 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <43E401E6-DC6D-46B2-94D4-7A961C177ED6@jlassocs.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <43E401E6-DC6D-46B2-94D4-7A961C177ED6@jlassocs.com> Message-ID: On Thu, Jul 5, 2012 at 11:01 AM, J E Lyon wrote: > On 5 Jul 2012, at 10:55, Kaya Saman wrote: > >> That's why I'm not even thinking of migrating the mission critical >> stuff running on CentOS 5 to even CentOS 6 yet. > > I'm in an identical position there -- and in fact, I think it's time to get some virtualised hosting of CentOS 6 servers, once I decide on the choice of underlying solution . . feels like a minefield at this point in time . . I've been working with RedHat then RH & Fedora then CentOS since the 90s, but haven't any firsthand experience of installing & maintaining VMWare or anything like it. Exciting times :) Wow it sounds nice :-) I am still part of the old school thought trend in that everything should have it's own dedicated hardware and preferably be SPARC cpu based, but if I took that attitude at work or anywhere else I went, I would be thought of as a troll :-P (use my way or else.... ?? blaaaah) > > >> Ok this may sound incredibly sad so don't sue me for it, but for my >> OpenSource work at home I have switched over from 15+ Linux servers >> down to 1x FreeBSD system running Jails. > > Ah yes... see above :) Vmware does not compare to FreeBSD and jails. Use FreeBSD becase blah blah blah.... haha sorry just thought I would inocently troll :-) Yeah I agree, Vmware or Citrix Xen do make life easier to accomodate for the broad spectrum user running a mixture of OS's - like most people. It is definitely the way forward. And as not to get too side tracked from the original content of the posting by the OP. It is worth making systems work and integrate together, meaning having a dedicated device for storage and then mounting that in whatever OS you run whichever service on. I am not sure though if a remote file system would need to be locally provisioned.... as if created a pool on a SAN or NAS with (x) file system. The Dovecot server shouldn't really need to know anything else other then where on that FS to put the Mail boxes or dir's. The only thing that I could think of is that disk IO or network bandwidth may be a performance factor but that should really only be it. Other then different file systems peforming in different ways of course. > > > J. Regards, Kaya From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 13:18:50 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 12:18:50 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF51A14.70102@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: > It absolutely kills me every time I see a mail server admin display > almost total lack of knowledge of his/her storage back end, or the > inability to describe it technically, in an email... You should get used to this. Welcome in XXI century! The rule is amount of real knowledge*official paper confirmed skills=const From CMarcus at Media-Brokers.com Thu Jul 5 13:33:08 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 05 Jul 2012 06:33:08 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: <4FF56D64.1020704@Media-Brokers.com> On 2012-07-05 5:45 AM, Kaya Saman wrote: > FreeBSD 8.2 x64 running on VMware Hi Kaya, Do you (or anyone else) know of any decent VMWare images (appliance) of current version of FreeBSD? I've been debating on switching from Gentoo to FreeBSD for a while now, and would love to find a ready made appliance (just basic uncustomized server install) that I could start with... Thanks, -- Best regards, Charles From h.reindl at thelounge.net Thu Jul 5 13:37:26 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 05 Jul 2012 12:37:26 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF56D64.1020704@Media-Brokers.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> Message-ID: <4FF56E66.3010007@thelounge.net> Am 05.07.2012 12:33, schrieb Charles Marcus: > On 2012-07-05 5:45 AM, Kaya Saman wrote: >> FreeBSD 8.2 x64 running on VMware > > Hi Kaya, > > Do you (or anyone else) know of any decent VMWare images (appliance) of current version of FreeBSD? I've been > debating on switching from Gentoo to FreeBSD for a while now, and would love to find a ready made appliance (just > basic uncustomized server install) that I could start with... do you really think it is a good idea to start with a pre-installed FREE operating system instead doing a fresh install? for testing maybe for production not really -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From kayasaman at gmail.com Thu Jul 5 13:38:31 2012 From: kayasaman at gmail.com (Kaya Saman) Date: Thu, 5 Jul 2012 11:38:31 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF56D64.1020704@Media-Brokers.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> Message-ID: On Thu, Jul 5, 2012 at 11:33 AM, Charles Marcus wrote: > On 2012-07-05 5:45 AM, Kaya Saman wrote: >> >> FreeBSD 8.2 x64 running on VMware > > > Hi Kaya, > > Do you (or anyone else) know of any decent VMWare images (appliance) of > current version of FreeBSD? I've been debating on switching from Gentoo to > FreeBSD for a while now, and would love to find a ready made appliance (just > basic uncustomized server install) that I could start with... > > Thanks, > > -- > > Best regards, > > Charles Hi Charles, this is actually offtopic from the OP however, feel free to PM with any questions you have :-) However, in response I didn't use any images, just the simple FreeBSD 8.2 AMD64 ISO..... and installed from there. Regards, Kaya From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 13:42:05 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 12:42:05 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> Message-ID: > > this is actually offtopic from the OP however, feel free to PM with > any questions you have :-) > > However, in response I didn't use any images, just the simple FreeBSD > 8.2 AMD64 ISO..... and installed from there. and it will work on many VM systems. And works best without any VM overlay. From kiwi at oav.net Thu Jul 5 13:51:59 2012 From: kiwi at oav.net (Xavier Beaudouin) Date: Thu, 05 Jul 2012 12:51:59 +0200 Subject: [Dovecot] Dovecot mailstorage migration uppon logging / delivery with dsync Message-ID: <734ee6d0f77fd78b719ee23659d96b50@oav.net> Hi there, I have a pretty standard setup with "simple" dovecot 2,0 configuration (eg /var/mailboxes on NFS with old netapp) and local ldap directory and Maildir. I wanted to find a good way to migrate from NFS old netapp to NFS new server (this one is not a netapp !). There 3 kind way (according to my own experience) - rsync old spool to new spool being patient stop services and then doing last rsync and mount /var/mailboxes mount point from old netapp to new nfs server). - doing imap_login (and other stuff like this) doing an rsync per box AFTER doing the first rsync with all boxes... - using dsync to fetch the maildir stuff from old server to new one, doing a simple flag, and doing "au fil de l'eau" migration. The last one seems to me a good idea, but I cannot figure host to make this correctly. Any hints or do someone here has already done this kind of stuff? Kind regards, -- Xavier Beaudouin From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 14:31:54 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 13:31:54 +0200 (CEST) Subject: [Dovecot] Dovecot mailstorage migration uppon logging / delivery with dsync In-Reply-To: <734ee6d0f77fd78b719ee23659d96b50@oav.net> References: <734ee6d0f77fd78b719ee23659d96b50@oav.net> Message-ID: > > I wanted to find a good way to migrate from NFS old netapp to NFS new server > (this one is not a netapp !). why not just attaching disks directly? going from older nonsense to newer one :) > There 3 kind way (according to my own experience) > > - rsync old spool to new spool being patient stop services and then doing > last rsync and mount /var/mailboxes mount point from old netapp to new nfs > server). right. this is the right way. last rsync should go quickly. well... not that quickly over NFS ;) From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 14:35:00 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 13:35:00 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> Message-ID: >> >> ~ James. > > Thank you, this is a valid suggestion, especially since it could be > done directly with some SQL magic in dovecot config file. > I will consider this option ! > as i always run dovecot using standard unix auth/password mechanism and mail user is always unix user, then it is rather trivial. editing automatically /etc/master.passwd to change home directories depending of your needs and then moving data is simple and you may do any way. Even without this i would recommend your way of spreading users by human-understandable way. i would take 2 first letters, mak groups out of them to have roughly same mount of users or disk space and then move. From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 14:35:45 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 13:35:45 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: > Am however, trying to do all clean installs on FreeBSD where I **can > ** get away with it. right. > > > Ok this may sound incredibly sad so don't sue me for it, but for my > OpenSource work at home I have switched over from 15+ Linux servers > down to 1x FreeBSD system running Jails. quite a common case. From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 14:36:15 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 13:36:15 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF55FCD.6070001@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: > > At 16TB+ scale with maildir you should be using XFS on kernel 3.x, not > EXT4. Your performance will be significantly better, as in 30% or much why you want to make 16TB partition at first place? From amateo at um.es Thu Jul 5 15:55:15 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 05 Jul 2012 14:55:15 +0200 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: References: <4FF49031.60701@um.es> Message-ID: <4FF58EB3.2020002@um.es> El 04/07/12 23:55, Timo Sirainen escribi?: > On 4.7.2012, at 21.49, Angel L. Mateo wrote: > >> Although nfs configuration is the same, there are a lot of differences on readdir vs readdirplus nfs operations. In fact, in the old one we have 12% readdir operations and 3% of readdirplus. And in the new one we have 46% of readdirplus and no readdir operations. > > I'm not entirely sure, but I think it's the kernel that decides if readdir or readdirplus is used and Dovecot can't affect that decision. (Unless maybe kernel does some heuristics.) > I think you are right. I guess that the differnce between my old system (who uses readdir and readdirplus) and the new one (which just uses readdirplus) is because of changes on the kernel. For some threads I have read (for example, http://www.spinics.net/lists/linux-nfs/msg19957.html) it seems that from some version in kernel source (2.6.36 if I remmeber well) some nfs improvements related with readdir and readdirplus (that makes nfs client only uses readdirplus when it was better) and recents kernels always use readdirplus. Concluding... in my systems, with users with hugh mail folders (thousands of mails) in maildir format, disabling rdirplus with mount options (mount option nordirplus) increases performance (maybe we could do more precise test and this could be documented in dovecot's nfs documentation) -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tss at iki.fi Thu Jul 5 16:21:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Jul 2012 16:21:03 +0300 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <4FF58EB3.2020002@um.es> References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> Message-ID: <22C7CFA5-C20A-4AE4-A0BA-00559C061DB7@iki.fi> On 5.7.2012, at 15.55, Angel L. Mateo wrote: > I think you are right. I guess that the differnce between my old system (who uses readdir and readdirplus) and the new one (which just uses readdirplus) is because of changes on the kernel. For some threads I have read (for example, http://www.spinics.net/lists/linux-nfs/msg19957.html) it seems that from some version in kernel source (2.6.36 if I remmeber well) some nfs improvements related with readdir and readdirplus (that makes nfs client only uses readdirplus when it was better) and recents kernels always use readdirplus. > > Concluding... in my systems, with users with hugh mail folders (thousands of mails) in maildir format, disabling rdirplus with mount options (mount option nordirplus) increases performance (maybe we could do more precise test and this could be documented in dovecot's nfs documentation) If readdirplus requires filesystem to do stat()s on each file, then that's definitely bad and not something Dovecot needs. The only use is when listing files in ~/Maildir/ and determining which ones are directories and which ones are files. But even that defaults to assuming that everything beginning with "." is a directory if the readdir() call didn't specify it, and in those systems where this isn't true you can set maildir_stat_dirs=yes setting. From thlabse at hotmail.com Thu Jul 5 16:45:12 2012 From: thlabse at hotmail.com (Tony Hlabse) Date: Thu, 5 Jul 2012 09:45:12 -0400 Subject: [Dovecot] LDAP.conf filter issue Message-ID: All, We are trying to use the user_filter options to limit authentication to a single user group. The cusotmer has security rules that they want to only have users belonging to a group to have access. Sample we have tried. auth_bind = yes auth_bind_userdn = EXAMPLE\%n base = / hosts = example.test.com ldap_version = 3 user_filter = (&(sAMAccountName=)(memberOf=CN=LR3BCC-DL,OU=Autonomy,OU=Exchange,OU=Enterprise Administration,DC=example,DC=test,DC=com)) Thanks Tony From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 17:40:28 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 16:40:28 +0200 (CEST) Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <4FF58EB3.2020002@um.es> References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> Message-ID: > (thousands of mails) in maildir format, this is small folders. From my practice average are 10000-20000 mails. Huge are over 150000. From tss at iki.fi Thu Jul 5 17:46:12 2012 From: tss at iki.fi (Timo Sirainen) Date: Thu, 5 Jul 2012 17:46:12 +0300 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <4FF58EB3.2020002@um.es> References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> Message-ID: <43E71D8A-4897-46A2-AF96-E21053023FC5@iki.fi> On 5.7.2012, at 15.55, Angel L. Mateo wrote: > Concluding... in my systems, with users with hugh mail folders (thousands of mails) in maildir format, disabling rdirplus with mount options (mount option nordirplus) increases performance (maybe we could do more precise test and this could be documented in dovecot's nfs documentation) BTW. What NFS server are you using? The problem is mainly on the server side, so would be useful to know which NFS server implementations (if any?) can handle readdirplus without such a huge performance loss. Mainly I'm wondering about NetApp, since that's what most large Dovecot NFS installations use. From mlopez at gtdinternet.com Thu Jul 5 18:11:51 2012 From: mlopez at gtdinternet.com (=?ISO-8859-1?Q?Mauricio_L=F3pez_Riffo?=) Date: Thu, 05 Jul 2012 11:11:51 -0400 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <43E71D8A-4897-46A2-AF96-E21053023FC5@iki.fi> References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> <43E71D8A-4897-46A2-AF96-E21053023FC5@iki.fi> Message-ID: <4FF5AEB7.2010003@gtdinternet.com> Timo, For Netapp NFS mailstorage, what options can use for mount point nfs? for better perfomance with huge maildir folders? (most of my customers use pop3 protocol instead imap) Any kernel customize? All of my mta servers is Centos 6 Regards El 05-07-2012 10:46, Timo Sirainen escribi?: > On 5.7.2012, at 15.55, Angel L. Mateo wrote: > >> Concluding... in my systems, with users with hugh mail folders (thousands of mails) in maildir format, disabling rdirplus with mount options (mount option nordirplus) increases performance (maybe we could do more precise test and this could be documented in dovecot's nfs documentation) > BTW. What NFS server are you using? The problem is mainly on the server side, so would be useful to know which NFS server implementations (if any?) can handle readdirplus without such a huge performance loss. Mainly I'm wondering about NetApp, since that's what most large Dovecot NFS installations use. > > -- Mauricio L?pez Riffo Red Hat Certified Engineer 804006455319519 Administrador de Servicios Internet ?rea Ingenier?a Gtd Internet S.A. http://www.grupogtd.com/ Moneda 920, Oficina 602 - Fono : +562 4139742 From list at airstreamcomm.net Thu Jul 5 19:28:15 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Thu, 05 Jul 2012 11:28:15 -0500 Subject: [Dovecot] Dsync replication Message-ID: <4FF5C09F.2060001@airstreamcomm.net> I have been loosely following discussions dsync replication, but I am wondering if this tool still in a testing phase or has it been committed as a production part of Dovecot? I would like to do some testing with the protocol to see if it's capable of handling the change rate our mail service generates, which has about 23,000 active users. We are trying to find solutions for a two site setup where mail processing is either being done at the primary data center A and fails over to a hot standby cluster at the secondary data center B, or where both data centers are doing active processing. I have been unable to find documentation on dsync replication on the wiki, so if there is any documentation available on how to setup dsync replication I would appreciate a nudge in the right direction. From wojtek at wojtek.tensor.gdynia.pl Thu Jul 5 19:35:25 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 5 Jul 2012 18:35:25 +0200 (CEST) Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> Message-ID: > > this is small folders. From my practice average are 10000-20000 mails. Huge > are over 150000. > > did tests with artifically created million mail folder. except i needed to rise vsize limit of dovecot process in config , and except first index creation it works smooth with dovecot taking little CPU and (after other programs generated some VM pressure) very little RAM, with minimal pagein. roundcube webmail worked smooth including search, thunderbird was extremely slow, imap itself wasn't generating considerable load. From a.kostyrev at serverc.ru Fri Jul 6 06:08:54 2012 From: a.kostyrev at serverc.ru (=?koi8-r?B?68/T1NnSxdcg4czFy9PBzsTSIOHMxcvTxcXXyd4=?=) Date: Fri, 6 Jul 2012 14:08:54 +1100 Subject: [Dovecot] Dsync replication In-Reply-To: <4FF5C09F.2060001@airstreamcomm.net> References: <4FF5C09F.2060001@airstreamcomm.net> Message-ID: <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> use the search, Luke) http://www.dovecot.org/list/dovecot/2012-March/064512.html this thread was all that I needed to setup replication for testing. -----Original Message----- From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of list at airstreamcomm.net Sent: Friday, July 06, 2012 3:28 AM To: dovecot at dovecot.org Subject: [Dovecot] Dsync replication I have been loosely following discussions dsync replication, but I am wondering if this tool still in a testing phase or has it been committed as a production part of Dovecot? I would like to do some testing with the protocol to see if it's capable of handling the change rate our mail service generates, which has about 23,000 active users. We are trying to find solutions for a two site setup where mail processing is either being done at the primary data center A and fails over to a hot standby cluster at the secondary data center B, or where both data centers are doing active processing. I have been unable to find documentation on dsync replication on the wiki, so if there is any documentation available on how to setup dsync replication I would appreciate a nudge in the right direction. From stan at hardwarefreak.com Fri Jul 6 07:04:42 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 05 Jul 2012 23:04:42 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> Message-ID: <4FF663DA.9090504@hardwarefreak.com> On 7/5/2012 6:36 AM, Wojciech Puchar wrote: >> >> At 16TB+ scale with maildir you should be using XFS on kernel 3.x, not >> EXT4. Your performance will be significantly better, as in 30% or much > > why you want to make 16TB partition at first place? You wouldn't partition the large LUN. You'd simply directly format it with XFS. Laying a partition table on it would introduce the real possibility of filesystem misalignment on the RAID stripe if one wasn't extremely careful about the partitioning alignment. But your question seems to relate to reason for using a single very large filesystem. In which case the answer is typically a single file tree/name space. EXT3/4 and Reiser apparently aren't super reliable at very large sizes. XFS has no such negative issues at large scale, and in fact was specifically designed for extremely large scale. As I mentioned previously, 50TB+ XFS filesystems are mundane and there are many 100TB+ in the wild. As with any small scale filesystem, you'll need a backup system capable of handling your very large XFS filesystem and a proper strategy. -- Stan From f.bonnet at esiee.fr Fri Jul 6 08:52:41 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Fri, 06 Jul 2012 07:52:41 +0200 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <4FF5AEB7.2010003@gtdinternet.com> References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> <43E71D8A-4897-46A2-AF96-E21053023FC5@iki.fi> <4FF5AEB7.2010003@gtdinternet.com> Message-ID: <4FF67D29.7030606@esiee.fr> I don't think there are NFS special options on NetApp filers but you can use ethernet jumbo frames to boost network access between your server and the filer. You also can use iSCSI between server and filer but it will waste a lot of disk space on the Ontap volume. I use NFS between a FreeBSD server and a 3210 netapp I use MBOX format , inboxes are living on the local raid array of the server and IMAP folders are located on a NetApp dedicated volume NFS mounted. The netapp filler has an amazing capacity to support very high load due to special cache hardware (nvram) we have only one 3210 that support all users's data + softwares of our users (2000) it never failed on slowdown accesses. Le 05/07/2012 17:11, Mauricio L?pez Riffo a ?crit : > Timo, > > For Netapp NFS mailstorage, what options can use for mount point > nfs? for better perfomance with huge maildir folders? (most of my > customers use pop3 protocol instead imap) Any kernel customize? All > of my mta servers is Centos 6 > > Regards > > > El 05-07-2012 10:46, Timo Sirainen escribi?: >> On 5.7.2012, at 15.55, Angel L. Mateo wrote: >> >>> Concluding... in my systems, with users with hugh mail folders >>> (thousands of mails) in maildir format, disabling rdirplus with >>> mount options (mount option nordirplus) increases performance (maybe >>> we could do more precise test and this could be documented in >>> dovecot's nfs documentation) >> BTW. What NFS server are you using? The problem is mainly on the >> server side, so would be useful to know which NFS server >> implementations (if any?) can handle readdirplus without such a huge >> performance loss. Mainly I'm wondering about NetApp, since that's >> what most large Dovecot NFS installations use. >> >> > From amateo at um.es Fri Jul 6 10:06:00 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 06 Jul 2012 09:06:00 +0200 Subject: [Dovecot] dovecot and nfs readdir vs readdirplus operations In-Reply-To: <43E71D8A-4897-46A2-AF96-E21053023FC5@iki.fi> References: <4FF49031.60701@um.es> <4FF58EB3.2020002@um.es> <43E71D8A-4897-46A2-AF96-E21053023FC5@iki.fi> Message-ID: <4FF68E58.6040401@um.es> El 05/07/12 16:46, Timo Sirainen escribi?: > On 5.7.2012, at 15.55, Angel L. Mateo wrote: > >> Concluding... in my systems, with users with hugh mail folders (thousands of mails) in maildir format, disabling rdirplus with mount options (mount option nordirplus) increases performance (maybe we could do more precise test and this could be documented in dovecot's nfs documentation) > > BTW. What NFS server are you using? The problem is mainly on the server side, so would be useful to know which NFS server implementations (if any?) can handle readdirplus without such a huge performance loss. Mainly I'm wondering about NetApp, since that's what most large Dovecot NFS installations use. > We are using a celerra from EMC, conected to an EMC SAN. -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n _o) y las Comunicaciones Aplicadas (ATICA) / \\ http://www.um.es/atica _(___V Tfo: 868887590 Fax: 868888337 From wojtek at wojtek.tensor.gdynia.pl Fri Jul 6 10:16:29 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 6 Jul 2012 09:16:29 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF663DA.9090504@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> Message-ID: > > You wouldn't partition the large LUN. You'd simply directly format it > with XFS. Laying a partition table on it would introduce the real Fine. i understand that. What i am suggesting is not making large LUNs. you get the best performance with directly attaching disks to your machine. From CMarcus at Media-Brokers.com Fri Jul 6 12:26:23 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 06 Jul 2012 05:26:23 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF56E66.3010007@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> <4FF56E66.3010007@thelounge.net> Message-ID: <4FF6AF3F.20003@Media-Brokers.com> On 2012-07-05 6:37 AM, Reindl Harald wrote: > Am 05.07.2012 12:33, schrieb Charles Marcus: >> On 2012-07-05 5:45 AM, Kaya Saman wrote: >>> FreeBSD 8.2 x64 running on VMware >> Do you (or anyone else) know of any decent VMWare images >> (appliance) of current version of FreeBSD? I've been debating on >> switching from Gentoo to FreeBSD for a while now, and would love to >> find a ready made appliance (just basic uncustomized server >> install) that I could start with... > do you really think it is a good idea to start with a pre-installed > FREE operating system instead doing a fresh install? do you really think it is a good idea to trash someone else's comments (without contributing anything at all I might add) based on pure ass-u-me-ptions of yours that have no basis in reality? > for testing maybe > for production not really Who said I was going to base my *production* server on such an image? I was asking for something so I could easily get started testing and playing with it. Do you *really* think I would simply start with a pre-built image and just switch my production to it without *lots* of testing (and getting used to the new environment, etc)? From h.reindl at thelounge.net Fri Jul 6 12:46:30 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 06 Jul 2012 11:46:30 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF6AF3F.20003@Media-Brokers.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> <4FF56E66.3010007@thelounge.net> <4FF6AF3F.20003@Media-Brokers.com> Message-ID: <4FF6B3F6.7040501@thelounge.net> Am 06.07.2012 11:26, schrieb Charles Marcus: > On 2012-07-05 6:37 AM, Reindl Harald wrote: >> do you really think it is a good idea to start with a pre-installed >> FREE operating system instead doing a fresh install? > > do you really think it is a good idea to trash someone else's comments (without contributing anything at all I > might add) based on pure ass-u-me-ptions of yours that have no basis in reality? where do you see anything offending in my reply? >> for testing maybe >> for production not really > > Who said I was going to base my *production* server on such an image? i do not know but why use it at all if it is not intentent to use later? > I was asking for something so I could easily get started testing and playing with it and in my opinion a pre-built image is the wrong start you learn much more by installing and configure any OS by yourself as by use a pre-installed without have a clue why things are working and how they do -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From CMarcus at Media-Brokers.com Fri Jul 6 13:01:46 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 06 Jul 2012 06:01:46 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF6B3F6.7040501@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> <4FF56E66.3010007@thelounge.net> <4FF6AF3F.20003@Media-Brokers.com> <4FF6B3F6.7040501@thelounge.net> Message-ID: <4FF6B78A.5080606@Media-Brokers.com> On 2012-07-06 5:46 AM, Reindl Harald wrote: > where do you see anything offending in my reply? Your tone is almost always offending, Reindl - and you quite often throw in a good dose of very offending cursing to boot (admittedly not this time though)... basically, I just don't like your general tone, it is *never* friendly, and more often than not it is unhelpful (like this time). > but why use it at all if it is not intentent to use later? To get up and running quickly and do some quick testing to see if i want to invest more serious time on it? The last time I tried to install freebsd from scratch (version 4/5 days) it was a horrendous episode that turned me off to freebsd right away. I want to avoid a repeat of that and start with a *working* system to see what i may have been missing. Seriously, for questions like this, either post a responsive answer, or just don't answer at all. > and in my opinion a pre-built image is the wrong start You know what they say about opinions... > you learn much more by installing and configure any OS by yourself as > by use a pre-installed without have a clue why things are working and > how they do Reindl - i'm a gentoo user, I know all about the advantages of installing from scratch. Again - just please stay silent if you don't have anything positive to contribute - and yes, you often do actually contribute positive things, and you definitely have some knowledge to share, but again, your tone and manner are almost always those of a know-it-all spoiled teenage brat, and I personally have had enough, hence my possibly overly harsh response to your know-it-all-non-response. From h.reindl at thelounge.net Fri Jul 6 13:16:14 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 06 Jul 2012 12:16:14 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF6B78A.5080606@Media-Brokers.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> <4FF56E66.3010007@thelounge.net> <4FF6AF3F.20003@Media-Brokers.com> <4FF6B3F6.7040501@thelounge.net> <4FF6B78A.5080606@Media-Brokers.com> Message-ID: <4FF6BAEE.6020405@thelounge.net> Am 06.07.2012 12:01, schrieb Charles Marcus: > Again - just please stay silent if you don't have anything positive to contribute - and yes, you often do actually > contribute positive things, and you definitely have some knowledge to share, but again, your tone and manner are > almost always those of a know-it-all spoiled teenage brat, and I personally have had enough, hence my possibly > overly harsh response to your know-it-all-non-response. interesting that with the same tone in real life in more than 30 years nobody had a problem as long it was not my intention however - i googled "freebsd vmware image" for you the first link leads to http://www.thoughtpolice.co.uk/vmware/ http://www.thoughtpolice.co.uk/vmware/#freebsd in VMware vCenter yxou can installa appliance directly while it is downloaded via the second entry in the file-menu -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From hansg at dandy.nl Fri Jul 6 16:09:28 2012 From: hansg at dandy.nl (Hans de Groot) Date: Fri, 06 Jul 2012 15:09:28 +0200 Subject: [Dovecot] Dovecot v2 shared mailboxes , a . separator and complete emailaddress as username. Message-ID: <2460-4ff6e380-9-2a9a3880@20300067> Hello, I've been trying to get shared mailboxes to work and got stuck. I tried lots of examples made the shared dict but the shared mail boxes never showed up. I am using the . separator and I think this the problem. I changed it to a / separator and now I could see my shared mailboxes. Checking the maillog in debug mode I saw a few failed login attempts between the succesful logins. The weird thing is that the username was changed, instead of hansg at debushalte.nl the login was hansg at debushalte without the .nl part and that user is not in my database. So I suspect that dovecot splits the shared path based hansg at debushalte.nl shares mailboxes with dean at debushalte.nl below a few lines from the log: Debug: Namespace : type=shared, prefix=INBOX.shared.%u., sep=., inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:%h/mail:INDEX=/mail/debushalte.nl/dean/mail/shared.%u Debug: shared: Tried to access mails of nonexistent user hansg at debushalte Debug: Namespace INBOX.shared.hansg at debushalte.: Creating storage despite: Root mail directory doesn't exist Is there a work arround for this? I really like to keep using '.' as a seperator so I do not have to change all the existing mailboxes. If I do rename and move all existing mailboxes the '/' separator way and of course change the separator in the namespaces would my users notice this? (ie would they have to change something in their mail clients?) Regards Hans From hansg at dandy.nl Fri Jul 6 16:46:36 2012 From: hansg at dandy.nl (Hans de Groot) Date: Fri, 06 Jul 2012 15:46:36 +0200 Subject: [Dovecot] I changed the separator from . to / and it still seems to work. Message-ID: <2460-4ff6ec00-b-2a9a3880@20300068> Hello, Just as an experiment I changed the separator from . to / restarted dovecot (v2.1.1). When adding an existing account with lots of mail folders to thunderbird everything still seems to work. My directory layout is like this: .Info.mailinglists .Info.mailinglists.Horde .Info.mailinglists.modperl .Info.mailinglists.PDNS .Info.mailinglists.Sogo .Info.mailinglists.tinc .Info.mailinglists.Zabbix I added a new folder to the horde mailing using thunderbird list and got an extra folder .Info.mailinglists.Horde.testinhorde Does this mean the separator has nothing to do with the . in the directory names and I am safe to switch saparators without having to do anything else? Regards Hans de Groot From dovecot at magic.360xl.net Fri Jul 6 17:46:24 2012 From: dovecot at magic.360xl.net (Vyacheslav Biruk) Date: Fri, 6 Jul 2012 14:46:24 +0000 Subject: [Dovecot] Pigeonhole Sieve SetFlag question Message-ID: <20120706144624.GA72888@chado.in.ua> Hello, I use FreeBSD 8.2-RELEASE-p7 i386 dovecot-2.1.6 dovecot-pigeonhole-0.3.0_1 I have the following problem: When I use the following script - the flag is set in \Seen : require ["imap4flags"]; # rule:[test] if anyof (header :contains "Subject" "test") { setflag "\\Seen"; stop; } . SELECT INBOX * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 1 RECENT * OK [UIDVALIDITY 1338600150] UIDs valid * OK [UIDNEXT 9] Predicted next UID * OK [NOMODSEQ] No permanent modsequences . OK [READ-WRITE] Select completed. . FETCH 1 (UID FLAGS BODY.PEEK[HEADER.FIELDS (DATE SUBJECT)]) * 1 FETCH (UID 8 FLAGS (\Seen \Recent) BODY[HEADER.FIELDS (DATE SUBJECT)] {57} Date: Fri, 6 Jul 2012 14:02:49 +0000 Subject: test 1 ) . OK Fetch completed. But when I use following script - the flag is not set in \Seen require ["fileinto","imap4flags"]; # rule:[test] if anyof (header :contains "Subject" "test") { fileinto "INBOX.SPAM"; setflag "\\Seen"; stop; } . SELECT INBOX.SPAM * OK [CLOSED] Previous mailbox closed. * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 1 EXISTS * 1 RECENT * OK [UNSEEN 1] First unseen. * OK [UIDVALIDITY 1338600159] UIDs valid * OK [UIDNEXT 7] Predicted next UID * OK [NOMODSEQ] No permanent modsequences . OK [READ-WRITE] Select completed. . FETCH 1 (UID FLAGS BODY.PEEK[HEADER.FIELDS (DATE SUBJECT)]) * 1 FETCH (UID 6 FLAGS (\Recent) BODY[HEADER.FIELDS (DATE SUBJECT)] {57} Date: Fri, 6 Jul 2012 13:53:29 +0000 Subject: test 1 ) . OK Fetch completed. so it should be or is it a bug? -- Best regards, magic From e-frog at gmx.de Fri Jul 6 18:10:40 2012 From: e-frog at gmx.de (e-frog) Date: Fri, 06 Jul 2012 17:10:40 +0200 Subject: [Dovecot] Pigeonhole Sieve SetFlag question In-Reply-To: <20120706144624.GA72888@chado.in.ua> References: <20120706144624.GA72888@chado.in.ua> Message-ID: <4FF6FFF0.3030302@gmx.de> On 06.07.2012 16:46, wrote Vyacheslav Biruk: > Hello, > > I use > FreeBSD 8.2-RELEASE-p7 i386 > dovecot-2.1.6 > dovecot-pigeonhole-0.3.0_1 > > I have the following problem: > But when I use following script - the flag is not set in \Seen > > require ["fileinto","imap4flags"]; > > # rule:[test] > if anyof (header :contains "Subject" "test") > { > fileinto "INBOX.SPAM"; > setflag "\\Seen"; > stop; > } This works for me... if anyof (header :contains "Subject" "test") { addflag "\\Seen"; fileinto "INBOX.SPAM"; stop; } From stephan at rename-it.nl Fri Jul 6 18:21:46 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 06 Jul 2012 17:21:46 +0200 Subject: [Dovecot] Pigeonhole Sieve SetFlag question In-Reply-To: <4FF6FFF0.3030302@gmx.de> References: <20120706144624.GA72888@chado.in.ua> <4FF6FFF0.3030302@gmx.de> Message-ID: <4FF7028A.8070900@rename-it.nl> Op 7/6/2012 5:10 PM, e-frog schreef: > On 06.07.2012 16:46, wrote Vyacheslav Biruk: > >> But when I use following script - the flag is not set in \Seen >> >> require ["fileinto","imap4flags"]; >> >> # rule:[test] >> if anyof (header :contains "Subject" "test") >> { >> fileinto "INBOX.SPAM"; >> setflag "\\Seen"; >> stop; >> } > > This works for me... > > if anyof (header :contains "Subject" "test") > { > addflag "\\Seen"; > fileinto "INBOX.SPAM"; > stop; > } Short story: put addflag/setflag/removeflag commands always before the fileinto command you want to affect. Long story: in this form, the addflag command manipulates an internal variable that keeps record of the active flags. Once the fileinto command is used, the status of that variable is used to set flags for the stored message. Manipulating the active flags after storing the message with fileinto has no effect on that command anymore. Only subsequent other fileinto commands will notice the change. Read RFC5232 (http://tools.ietf.org/html/rfc5232) for more info. Regards, Stephan. From wojtek at wojtek.tensor.gdynia.pl Fri Jul 6 19:41:51 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 6 Jul 2012 18:41:51 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF6AF3F.20003@Media-Brokers.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> <4FF56E66.3010007@thelounge.net> <4FF6AF3F.20003@Media-Brokers.com> Message-ID: > > do you really think it is a good idea to trash someone else's comments > (without contributing anything at all I might add) based on pure > ass-u-me-ptions of yours that have no basis in reality? Do you hate yourself of not being able to understand normal response and so - getting agressive against people? From simon.buongiorno at gmail.com Fri Jul 6 19:59:09 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Fri, 6 Jul 2012 12:59:09 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> <4FF56E66.3010007@thelounge.net> <4FF6AF3F.20003@Media-Brokers.com> Message-ID: On 6 July 2012 12:41, Wojciech Puchar wrote: >> >> do you really think it is a good idea to trash someone else's comments >> (without contributing anything at all I might add) based on pure >> ass-u-me-ptions of yours that have no basis in reality? > > > Do you hate yourself of not being able to understand normal response and so > - getting agressive against people? Pot. Kettle. Black. From list at airstreamcomm.net Fri Jul 6 23:28:38 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Fri, 06 Jul 2012 15:28:38 -0500 Subject: [Dovecot] Dsync replication In-Reply-To: <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> References: <4FF5C09F.2060001@airstreamcomm.net> <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> Message-ID: <4FF74A76.6010605@airstreamcomm.net> On 7/5/12 10:08 PM, ???????? ????????? ?????????? wrote: > use the search, Luke) > > http://www.dovecot.org/list/dovecot/2012-March/064512.html > this thread was all that I needed to setup replication for testing. > > > > -----Original Message----- > From: dovecot-bounces at dovecot.org [mailto:dovecot-bounces at dovecot.org] On Behalf Of list at airstreamcomm.net > Sent: Friday, July 06, 2012 3:28 AM > To: dovecot at dovecot.org > Subject: [Dovecot] Dsync replication > > I have been loosely following discussions dsync replication, but I am > wondering if this tool still in a testing phase or has it been committed > as a production part of Dovecot? I would like to do some testing with > the protocol to see if it's capable of handling the change rate our mail > service generates, which has about 23,000 active users. We are trying > to find solutions for a two site setup where mail processing is either > being done at the primary data center A and fails over to a hot standby > cluster at the secondary data center B, or where both data centers are > doing active processing. I have been unable to find documentation on > dsync replication on the wiki, so if there is any documentation > available on how to setup dsync replication I would appreciate a nudge > in the right direction. > > > Thanks, that certainly helps identify the configuration options. However I am more concerned about the experiences of others who have actually used the replication. What is the rate of change on your mail cluster, how many concurrent users do you support with replication enabled, do you use synchronous or asynchronous replication, are you using it in an active/active or active/passive state, is it possible to have a cluster with multiple servers at each site hosting the same mail data, does dysnc replication scale well (10,000 -> 100,000 -> 1,000,000 users)? Just trying to get a good feel for whether dsync replication is capable of handling the use case I am proposing before investing too much time in testing it. From jtam.home at gmail.com Sat Jul 7 03:38:23 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 6 Jul 2012 17:38:23 -0700 (PDT) Subject: [Dovecot] Dovecot 2.1.8 crashes when doing POP3 UIDL on empty INBOX Message-ID: I noticed a bunch of crashes when Gmail users tried to slurp up their (empty) mailboxes. The problem is not noticed by clients though, but it crashes the pop3 process. POP3 session S: +OK Ready. C: USER user S: +OK C: PASS password S: +OK Logged in. C: UIDL ... server crash and disconnects ... Resulting log Jul 06 16:58:05 pop3(user): Panic: Trying to allocate 0 bytes Jul 06 16:58:05 pop3(user): Error: Raw backtrace: 0xff1e6454 -> 0xff1e4f78 -> 0xff20689c -> 0x18240 -> 0x1843c - > 0x185a0 -> 0x18a1c -> 0x1576c -> 0x159cc -> 0xff1fee6c -> 0xff200e24 -> 0xff1fef60 -> 0xff1d8010 -> 0x13584 - > 0x1285c Jul 06 16:58:14 pop3(user): Fatal: master: service(pop3): child 24972 killed with signal 6 (core dumps disabled) GDB traceback: #0 i_panic (format=0xff2302f8 "Trying to allocate %u bytes") at failures.c:259 #1 0xff2068a4 in pool_alloconly_malloc (pool=0x60330, size=0) at mempool-alloconly.c:259 #2 0x00018248 in client_uidls_save (client=0x54d28) at pop3-commands.c:761 #3 0x00018444 in cmd_uidl_init (client=0x54d28, seq=0) at pop3-commands.c:793 #4 0x000185a8 in cmd_uidl (client=0x54d28, args=0x19eb8 "") at pop3-commands.c:824 #5 0x00018a24 in client_command_execute (client=0x54d28, name=0x2b550 "UIDL", args=0x19eb8 "") at pop3-commands.c:889 #6 0x00015774 in client_handle_input (client=0x54d28) at pop3-client.c:629 #7 0x000159d4 in client_input (client=0x54d28) at pop3-client.c:682 #8 0xff1fee74 in io_loop_call_io (io=0x37298) at ioloop.c:379 #9 0xff200e2c in io_loop_handler_run (ioloop=0x34138) at ioloop-poll.c:211 #10 0xff1fef68 in io_loop_run (ioloop=0x34138) at ioloop.c:398 #11 0xff1d8018 in master_service_run (service=0x33c88, callback=0x13120 ) at master-service.c:543 #12 0x0001358c in main (argc=1, argv=0xffbffe0c) at main.c:268 Some non-trivial changes in pop3-commands.c were done between 2.1.3 and 2.1.8 (expecially pop3_uidl_duplicates changes in 2.1.7). I guess this bug has crept in there. Joseph Tam From tss at iki.fi Sat Jul 7 05:52:03 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 7 Jul 2012 05:52:03 +0300 Subject: [Dovecot] Dovecot 2.1.8 crashes when doing POP3 UIDL on empty INBOX In-Reply-To: References: Message-ID: <6D7C5963-1918-40DF-9D70-78278B98CC52@iki.fi> On 7.7.2012, at 3.38, Joseph Tam wrote: > #0 i_panic (format=0xff2302f8 "Trying to allocate %u bytes") at failures.c:259 > #1 0xff2068a4 in pool_alloconly_malloc (pool=0x60330, size=0) at mempool-alloconly.c:259 > #2 0x00018248 in client_uidls_save (client=0x54d28) at pop3-commands.c:761 > #3 0x00018444 in cmd_uidl_init (client=0x54d28, seq=0) at pop3-commands.c:793 > #4 0x000185a8 in cmd_uidl (client=0x54d28, args=0x19eb8 "") at pop3-commands.c:824 > #5 0x00018a24 in client_command_execute (client=0x54d28, name=0x2b550 "UIDL", args=0x19eb8 "") > at pop3-commands.c:889 Fixed: http://hg.dovecot.org/dovecot-2.1/rev/ea18b2ddb67b From tss at iki.fi Sat Jul 7 05:56:25 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 7 Jul 2012 05:56:25 +0300 Subject: [Dovecot] Dsync replication In-Reply-To: <4FF74A76.6010605@airstreamcomm.net> References: <4FF5C09F.2060001@airstreamcomm.net> <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> <4FF74A76.6010605@airstreamcomm.net> Message-ID: <43111E61-9EA3-4D18-B55B-C0F40DD891BE@iki.fi> On 6.7.2012, at 23.28, list at airstreamcomm.net wrote: > Thanks, that certainly helps identify the configuration options. However I am more concerned about the experiences of others who have actually used the replication. What is the rate of change on your mail cluster, how many concurrent users do you support with replication enabled, do you use synchronous or asynchronous replication, are you using it in an active/active or active/passive state, is it possible to have a cluster with multiple servers at each site hosting the same mail data, does dysnc replication scale well (10,000 -> 100,000 -> 1,000,000 users)? Just trying to get a good feel for whether dsync replication is capable of handling the use case I am proposing before investing too much time in testing it. I wouldn't use it for large systems yet. It is still pretty inefficient. v2.2 will have a redesigned dsync that can do incremental syncs much faster and with less bandwidth. Anyway, in my small installation I'm using it in active-active mode and it works well enough. I've even configured my clients intentionally so that they use different servers. From stan at hardwarefreak.com Sat Jul 7 09:37:31 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sat, 07 Jul 2012 01:37:31 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> Message-ID: <4FF7D92B.6070401@hardwarefreak.com> On 7/6/2012 2:16 AM, Wojciech Puchar wrote: >> >> You wouldn't partition the large LUN. You'd simply directly format it >> with XFS. Laying a partition table on it would introduce the real > > Fine. i understand that. What i am suggesting is not making large LUNs. > you get the best performance with directly attaching disks to your machine. That's simply not true. 99% of block latency is rotational. iSCSI packet latency is an order of magnitude less than rotational latency, and fiber channel packet latency is a couple orders of magnitude lower still, even with a couple switches between the server and storage array. Thus it doesn't matter how you attach your mechanical disks to a server, as the latency is in the drives, not the data path. -- Stan From wojtek at wojtek.tensor.gdynia.pl Sat Jul 7 12:23:29 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sat, 7 Jul 2012 11:23:29 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF7D92B.6070401@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> Message-ID: >> Fine. i understand that. What i am suggesting is not making large LUNs. >> you get the best performance with directly attaching disks to your machine. > > That's simply not true. 99% of block latency is rotational. iSCSI It's not about iSCSI latency and overhead. It's about other things i just don't want to explain anymore for people that just have to much money to spent to make things more complex and less efficient. It's just stupid, but it's OT so EOT. From role.Dovecot-Readers at JLAssocs.com Sat Jul 7 12:27:50 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Sat, 7 Jul 2012 10:27:50 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF7D92B.6070401@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> Message-ID: <4285BE14-010A-48D8-AD3B-12125BA82C6B@JLAssocs.com> On 7 Jul 2012, at 07:37, Stan Hoeppner wrote: > 99% of block latency is rotational. So true... I spend my entire life trying to convince customers to add heaps and heaps of RAM to *nix servers to make them faster and not be swayed by talk of faster CPUs . . Sheeesh! . . Come to think of it, I'd dump enough RAM in my servers to keep the entire disk array cached if I could, um, well *coughs* ~ James. From h.reindl at thelounge.net Sat Jul 7 12:36:02 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 07 Jul 2012 11:36:02 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> Message-ID: <4FF80302.2020800@thelounge.net> Am 07.07.2012 11:23, schrieb Wojciech Puchar: >>> Fine. i understand that. What i am suggesting is not making large LUNs. >>> you get the best performance with directly attaching disks to your machine. >> >> That's simply not true. 99% of block latency is rotational. iSCSI > It's not about iSCSI latency and overhead. > > It's about other things i just don't want to explain anymore for people that > just have to much money to spent to make things more complex and less efficient. > It's just stupid, but it's OT so EOT what are you trying to tell us here? do you REALLY believe you local disks can outperform a SAN storage with 1 GB dedicated buffer cache und a DEDUCATED 1400 MHz CPU which is only optimized for one task: disk performance you lcal storage has to fight for CPU and memory all teh time with other applications (caching etc.) to believe under really high load a local storage is faster at the end is bullshit! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From noel.butler at ausics.net Sat Jul 7 12:39:45 2012 From: noel.butler at ausics.net (Noel Butler) Date: Sat, 07 Jul 2012 19:39:45 +1000 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> Message-ID: <1341653985.10382.12.camel@tardis> On Sat, 2012-07-07 at 11:23 +0200, Wojciech Puchar wrote: > >> Fine. i understand that. What i am suggesting is not making large LUNs. > >> you get the best performance with directly attaching disks to your machine. > > > > That's simply not true. 99% of block latency is rotational. iSCSI > It's not about iSCSI latency and overhead. > > It's about other things i just don't want to explain anymore for people > that just have to much money to spent to make things more complex and less > efficient. It's just stupid, but it's OT so EOT. You get used to Stan's rants, most ignore his drivel, been proved before he knows enough about hardware to be dangerous, and even less about anything bigger than SOHO mail networks. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From wojtek at wojtek.tensor.gdynia.pl Sat Jul 7 12:43:45 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sat, 7 Jul 2012 11:43:45 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF80302.2020800@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> Message-ID: > SAN storage with 1 GB dedicated buffer cache und a > DEDUCATED 1400 MHz CPU which is only optimized for > one task: disk performance > > you lcal storage has to fight for CPU and memory all > teh time with other applications (caching etc.) > > to believe under really high load a local storage > is faster at the end is bullshit! Yes i do believe in that "bullshit". As well as i believe that no amount of money for "professional hardware" will replace brain. From h.reindl at thelounge.net Sat Jul 7 13:00:48 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 07 Jul 2012 12:00:48 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> Message-ID: <4FF808D0.1090304@thelounge.net> Am 07.07.2012 11:43, schrieb Wojciech Puchar: >> SAN storage with 1 GB dedicated buffer cache und a >> DEDUCATED 1400 MHz CPU which is only optimized for >> one task: disk performance >> >> you lcal storage has to fight for CPU and memory all >> teh time with other applications (caching etc.) >> >> to believe under really high load a local storage >> is faster at the end is bullshit! > Yes i do believe in that "bullshit" you problem > As well as i believe that no amount of money for > "professional hardware" will replace brain this is a laughable argumentation because your brain can not be so much bigger than others because you will not show me as example VMware HA on a local storage and yes, tehre are very very few cases where it makes sense this days to not use virtualization for practically all if you have a SPECIAL workload that may be different but to believe you have more brain because you never were in the position to be responsible for ANY infrastructure component (hardware and osftware) is arrogant without any background -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From malloc4k at gmail.com Sat Jul 7 17:26:11 2012 From: malloc4k at gmail.com (Malloc Kilobyte) Date: Sat, 7 Jul 2012 16:26:11 +0200 Subject: [Dovecot] Feature request Message-ID: Hi, I am implementig Dovecot as a part of my new e-mail server, which is aimed to be replacement for proprietary software I've been using sofar. Appreciating all Dovecot rich features, I lack just one. And this is the ability to customize the "quota exceeded, message rejected" message. I know I can set it's default content using quota_exceeded_message parameter, but i would also like to have ability to set: - to whom the message is sent. Some mail systems send copy not only to the sender, but also to the recipient. It makes sense in case rejected message has big size. - it's subject - if the rejected message is being attached ( with or without it's own attachments ) or not Moreover, I think it would make sense if the "rejected messege" contained information about size of rejected message and current mailbox space usage. Please, consider this in future development. with regards and respect to the tremendousness of Dovecot, -- malloc4k From h.reindl at thelounge.net Sat Jul 7 17:35:45 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 07 Jul 2012 16:35:45 +0200 Subject: [Dovecot] Feature request In-Reply-To: References: Message-ID: <4FF84941.8040406@thelounge.net> Am 07.07.2012 16:26, schrieb Malloc Kilobyte: > Appreciating all Dovecot rich features, I lack just one. And this is the > ability to customize the "quota exceeded, message rejected" message. I know > I can set it's default content using quota_exceeded_message parameter, but > i would also like to have ability to set: > > - to whom the message is sent. Some mail systems send copy not only to the > sender, but also to the recipient. It makes sense in case rejected message > has big size. > > - it's subject > > - if the rejected message is being attached ( with or without it's own > attachments ) or not > > Moreover, I think it would make sense if the "rejected messege" contained > information about size of rejected message and current mailbox space usage. hmm - dovecot doe snot send any mail so there is not sobject and to whom the message is sen messages are typically only REJECTED eith status code and status message the mail itself is based on this jerecht code/message from the delivering MTA -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Sat Jul 7 18:15:36 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Sun, 8 Jul 2012 01:15:36 +1000 Subject: [Dovecot] + sub addressing setup In-Reply-To: <4FF47E6F.4030108@localhost.localdomain.org> References: <72fdefa5f56f58aa558c0f87c6606062.squirrel@sbt.net.au> <4FF1E1A3.3010202@localhost.localdomain.org> <4FF1E525.6020606@localhost.localdomain.org> <4934f87e7cfe3ac357d3694ecdacb9d3.squirrel@sbt.net.au> <4FF47E6F.4030108@localhost.localdomain.org> Message-ID: <0e4bf5cabffe5d57e82b8097fe3a01f7.squirrel@www.katyn.org.au> On Thu, July 5, 2012 3:33 am, Pascal Volk wrote: > I guess there is somewhere a missing closing curly bracket in one of > your configuration files. Pascal, thanks for all the pointers, I think I got it, I inserted some of the lines in dovecot.conf itself (rather than conf.d), and, I think it's working: ---------- # grep dovecot /var/log/maillog | grep lmtp Jul 8 01:05:41 dovecot: lmtp(30465): Connect from local Jul 8 01:05:41 dovecot: lmtp(30465, v at tld): XSqYMEVQ+E8BdwAAyLbbsQ: msgid=<860624821.1534673480413.JavaMail.app at rbg42.atlis1>: saved mail to INBOX Jul 8 01:05:41 postfix/lmtp[30623]: 984DB381909: to=, relay=host.tld[private/dovecot-lmtp], delay=0.33, delays=0.18/0.01/0/0.14, dsn=2.0.0, status=sent (250 2.0.0 XSqYMEVQ+E8BdwAAyLbbsQ Saved) Jul 8 01:05:41 dovecot: lmtp(30465): Disconnect from local: Client quit (in reset) --------- thanks very much for your help so, is this how it's supposed to work: as long as there is a corresponding directory 'subdir', name+subdir gets delivered to 'subdir', otherwise, if no 'subdir', gets delivered to inbox ? -- Voytek From arnaud.abelard at univ-nantes.fr Sat Jul 7 18:36:08 2012 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Sat, 07 Jul 2012 17:36:08 +0200 Subject: [Dovecot] Feature request In-Reply-To: <4FF84941.8040406@thelounge.net> References: <4FF84941.8040406@thelounge.net> Message-ID: <4FF85768.9010001@univ-nantes.fr> On 07/07/2012 04:35 PM, Reindl Harald wrote: > > > Am 07.07.2012 16:26, schrieb Malloc Kilobyte: >> Appreciating all Dovecot rich features, I lack just one. And this is the >> ability to customize the "quota exceeded, message rejected" message. I know >> I can set it's default content using quota_exceeded_message parameter, but >> i would also like to have ability to set: >> >> - to whom the message is sent. Some mail systems send copy not only to the >> sender, but also to the recipient. It makes sense in case rejected message >> has big size. >> >> - it's subject >> >> - if the rejected message is being attached ( with or without it's own >> attachments ) or not >> >> Moreover, I think it would make sense if the "rejected messege" contained >> information about size of rejected message and current mailbox space usage. > > hmm - dovecot doe snot send any mail > so there is not sobject and to whom the message is sen > > messages are typically only REJECTED eith status code and status message > > the mail itself is based on this jerecht code/message from the > delivering MTA Then the cool feature could be to be able to make the quota plugin call an external script/command which could do whatever the admins wish: Send an email to a backup account, modify a database, set off an alarm, phone the president, slap the user in the face or whatever. As far as I know that feature could already exist since the idea just popped to me and didn't bother checking the documentation yet. So if it is so, my bad. Arnaud -- Arnaud Ab?lard jabber: arnaud.abelard at univ-nantes.fr / twitter: ArnY Administrateur Syst?me DSI Universit? de Nantes - From wojtek at wojtek.tensor.gdynia.pl Sat Jul 7 18:43:19 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sat, 7 Jul 2012 17:43:19 +0200 (CEST) Subject: [Dovecot] Feature request In-Reply-To: References: Message-ID: > Appreciating all Dovecot rich features, I lack just one. And this is the > ability to customize the "quota exceeded, message rejected" message. I know no because SMTP mail delivery is not dovecot job. From lists at wildgooses.com Sat Jul 7 21:36:35 2012 From: lists at wildgooses.com (Ed W) Date: Sat, 07 Jul 2012 19:36:35 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF56D64.1020704@Media-Brokers.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF56D64.1020704@Media-Brokers.com> Message-ID: <4FF881B3.7050301@wildgooses.com> On 05/07/2012 11:33, Charles Marcus wrote: > On 2012-07-05 5:45 AM, Kaya Saman wrote: >> FreeBSD 8.2 x64 running on VMware > > Hi Kaya, > > Do you (or anyone else) know of any decent VMWare images (appliance) > of current version of FreeBSD? I've been debating on switching from > Gentoo to FreeBSD for a while now, and would love to find a ready made > appliance (just basic uncustomized server install) that I could start > with... We use Gentoo host + Linux-vservers (+grsec/pax) and very satisfied. Linux-vservers gives you something similar to "jails", although it's meant to look a little more like full virtualisation than jails does (bear in mind I don't have "jails" experience though) There are plenty of tools included with linux-vservers to clone, build and maintain your individual "machines". It's a complete enough "virtualisation" that you can say boot a centos image under your gentoo host (or whatever). It's also extremely lightweight, so there is almost zero overhead. It's a little weak on areas where you need direct access to hardware, but there are generally acceptable workarounds - also you can't run completely different operating systems since it's not a full virtualisation solution One nice benefit is that all images are just a directory containing your linux installation, so it's very easy to backup/snapshot/restore/drop in and fix something you bolloxed up/clone to a new machine. Just my 2p. Cheers Ed W From jtam.home at gmail.com Sun Jul 8 10:00:13 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Sun, 8 Jul 2012 00:00:13 -0700 (PDT) Subject: [Dovecot] Dovecot 2.1.8 crashes when doing POP3 UIDL on empty INBOX In-Reply-To: References: Message-ID: Timo Sirainen writes: >> #0 i_panic (format=0xff2302f8 "Trying to allocate %u bytes") at failures.c:259 >> #1 0xff2068a4 in pool_alloconly_malloc (pool=0x60330, size=0) at mempool-alloconly.c:259 >> #2 0x00018248 in client_uidls_save (client=0x54d28) at pop3-commands.c:761 >> #3 0x00018444 in cmd_uidl_init (client=0x54d28, seq=0) at pop3-commands.c:793 >> #4 0x000185a8 in cmd_uidl (client=0x54d28, args=0x19eb8 "") at pop3-commands.c:824 >> #5 0x00018a24 in client_command_execute (client=0x54d28, name=0x2b550 "UIDL", args=0x19eb8 "") >> at pop3-commands.c:889 > > Fixed: http://hg.dovecot.org/dovecot-2.1/rev/ea18b2ddb67b > > Date: Sat, 7 Jul 2012 05:52:03 +0300 2 hours after I reported it; do you ever sleep?! Thanks. Joseph Tam From role.Dovecot-Readers at JLAssocs.com Sun Jul 8 10:07:07 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Sun, 8 Jul 2012 08:07:07 +0100 Subject: [Dovecot] Dovecot 2.1.8 crashes when doing POP3 UIDL on empty INBOX In-Reply-To: References: Message-ID: <1734F789-EF17-43EC-8C97-E593C20FBD97@JLAssocs.com> On 8 Jul 2012, at 08:00, Joseph Tam wrote: > Timo Sirainen writes: > >>> #0 i_panic (format=0xff2302f8 "Trying to allocate %u bytes") at failures.c:259 >> >> Fixed: http://hg.dovecot.org/dovecot-2.1/rev/ea18b2ddb67b >> >> Date: Sat, 7 Jul 2012 05:52:03 +0300 > > 2 hours after I reported it; do you ever sleep?! Thanks. I think some of the things Timo does, he is able to do in his sleep . . (What *is* this thing 'sleep' of which you speak, anyway?) :) From tss at iki.fi Sun Jul 8 10:15:44 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 8 Jul 2012 10:15:44 +0300 Subject: [Dovecot] Dovecot 2.1.8 crashes when doing POP3 UIDL on empty INBOX In-Reply-To: <1734F789-EF17-43EC-8C97-E593C20FBD97@JLAssocs.com> References: <1734F789-EF17-43EC-8C97-E593C20FBD97@JLAssocs.com> Message-ID: On 8.7.2012, at 10.07, J E Lyon wrote: > I think some of the things Timo does, he is able to do in his sleep . . What's annoying is that I sometimes do, and all the hard work gets lost somewhere to the dream world :( From slitt at troubleshooters.com Sun Jul 8 10:27:55 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Sun, 8 Jul 2012 03:27:55 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF80302.2020800@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> Message-ID: <20120708032755.74728b69@mydesk> On Sat, 07 Jul 2012 11:36:02 +0200, Reindl Harald said: > > > Am 07.07.2012 11:23, schrieb Wojciech Puchar: > >>> Fine. i understand that. What i am suggesting is not making large > >>> LUNs. you get the best performance with directly attaching disks > >>> to your machine. > >> > >> That's simply not true. 99% of block latency is rotational. iSCSI > > It's not about iSCSI latency and overhead. > > > > It's about other things i just don't want to explain anymore for > > people that just have to much money to spent to make things more > > complex and less efficient. It's just stupid, but it's OT so EOT > > what are you trying to tell us here? > > do you REALLY believe you local disks can outperform a > SAN storage with 1 GB dedicated buffer cache und a > DEDUCATED 1400 MHz CPU which is only optimized for > one task: disk performance > > you lcal storage has to fight for CPU and memory all > teh time with other applications (caching etc.) > > to believe under really high load a local storage > is faster at the end is bullshit! Can one even argue on one side or the other without knowing the speed of the network, and how much contention is on that network? My experience is that with a 100Mbs network, local is faster, although I've never had a SAN, so to speak, on the other end. The specification of a SATA rev 3 is 6Gbs, which is a heck of a lot faster than 1Gbs per second spec of a gigabit network. Both have a lot of things slowing them from their spec, but I'd need to see some proof of an assertion that anything coming in over a 1Gbs wire can beat a SATA rev3 local disk. Thanks SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From slitt at troubleshooters.com Sun Jul 8 10:36:54 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Sun, 8 Jul 2012 03:36:54 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <20120708032755.74728b69@mydesk> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> Message-ID: <20120708033654.0709ba04@mydesk> On Sun, 8 Jul 2012 03:27:55 -0400, Steve Litt said: > On Sat, 07 Jul 2012 11:36:02 +0200, Reindl Harald said: > > > > > > Am 07.07.2012 11:23, schrieb Wojciech Puchar: > > >>> Fine. i understand that. What i am suggesting is not making > > >>> large LUNs. you get the best performance with directly > > >>> attaching disks to your machine. > > >> > > >> That's simply not true. 99% of block latency is rotational. > > >> iSCSI > > > It's not about iSCSI latency and overhead. > > > > > > It's about other things i just don't want to explain anymore for > > > people that just have to much money to spent to make things more > > > complex and less efficient. It's just stupid, but it's OT so EOT > > > > what are you trying to tell us here? > > > > do you REALLY believe you local disks can outperform a > > SAN storage with 1 GB dedicated buffer cache und a > > DEDUCATED 1400 MHz CPU which is only optimized for > > one task: disk performance > > > > you lcal storage has to fight for CPU and memory all > > teh time with other applications (caching etc.) > > > > to believe under really high load a local storage > > is faster at the end is bullshit! > > Can one even argue on one side or the other without knowing the speed > of the network, and how much contention is on that network? > > My experience is that with a 100Mbs network, local is faster, although > I've never had a SAN, so to speak, on the other end. > > The specification of a SATA rev 3 is 6Gbs, which is a heck of a lot > faster than 1Gbs per second spec of a gigabit network. Both have a lot > of things slowing them from their spec, but I'd need to see some proof > of an assertion that anything coming in over a 1Gbs wire can beat a > SATA rev3 local disk. This isn't really a fair comparision because I don't think rev3 is commodity yet. So Rev 2, which IS commodity, is 3Gbs, which is still considerably faster than the wire on a Gigabit network. Thanks SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From role.Dovecot-Readers at JLAssocs.com Sun Jul 8 11:28:43 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Sun, 8 Jul 2012 09:28:43 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <20120708033654.0709ba04@mydesk> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> Message-ID: <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> On 8 Jul 2012, at 08:36, Steve Litt wrote: >> Can one even argue on one side or the other without knowing the speed >> of the network, and how much contention is on that network? >> >> My experience is that with a 100Mbs network, local is faster, although >> I've never had a SAN, so to speak, on the other end. >> >> The specification of a SATA rev 3 is 6Gbs, which is a heck of a lot >> faster than 1Gbs per second spec of a gigabit network. Both have a lot >> of things slowing them from their spec, but I'd need to see some proof >> of an assertion that anything coming in over a 1Gbs wire can beat a >> SATA rev3 local disk. > > This isn't really a fair comparision because I don't think rev3 is > commodity yet. So Rev 2, which IS commodity, is 3Gbs, which is still > considerably faster than the wire on a Gigabit network. I think there are optimal situations where any configuration looks good . . How often can a real-world disk actually deliver the 6Gbs when only a minority of disk reads are long sequential runs on the platters? That's why I take the broader view . . over the course of 2-3 years, say, with a range of applications and demands, the total impact of resources, reliability, management and end-user experience with large systems that are large and complex by virtue of demand rather than through any fault of design, may be better served by a high-performance storage solution connected with extremely high-speed dedicated channels (I don't think anyone ever suggested 100mbps network for SAN was a high-performance scenario) . . and then look at how things work overall, including hardware maintenance for example. So, the laboratory / theoretical throughput of an internal 6Gbs bus is only partly a factor, imho... From wojtek at wojtek.tensor.gdynia.pl Sun Jul 8 11:38:18 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sun, 8 Jul 2012 10:38:18 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> Message-ID: > I think there are optimal situations where any configuration looks good . . How often can a real-world disk actually deliver the 6Gbs when only a minority of disk reads are long sequential runs on the platters? none of hard drives can saturate 1.5Gb/s From h.reindl at thelounge.net Sun Jul 8 12:55:40 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sun, 08 Jul 2012 11:55:40 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <20120708032755.74728b69@mydesk> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> Message-ID: <4FF9591C.20501@thelounge.net> Am 08.07.2012 09:27, schrieb Steve Litt: > On Sat, 07 Jul 2012 11:36:02 +0200, Reindl Harald said: >> to believe under really high load a local storage >> is faster at the end is bullshit! > > Can one even argue on one side or the other without knowing the speed > of the network, and how much contention is on that network? the SAN has normally it's own network > My experience is that with a 100Mbs network, local is faster, although > I've never had a SAN, so to speak, on the other end. nobody is using 100 MBit for a SAN > The specification of a SATA rev 3 is 6Gbs, which is a heck of a lot > faster than 1Gbs per second spec of a gigabit network. Both have a lot > of things slowing them from their spec, but I'd need to see some proof > of an assertion that anything coming in over a 1Gbs wire can beat a > SATA rev3 local disk. there is more than the connection speed 6Gbsdo not help you much as long the physical disk can not write in this speed and more concurretn writes making this worser - so there are many things like big battery backed caches fon a SAN which are imprtant for OVERALL performance -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From malloc4k at gmail.com Sun Jul 8 13:20:56 2012 From: malloc4k at gmail.com (Malloc Kilobyte) Date: Sun, 8 Jul 2012 12:20:56 +0200 Subject: [Dovecot] Feature request In-Reply-To: <4FF84941.8040406@thelounge.net> References: <4FF84941.8040406@thelounge.net> Message-ID: On Sat, Jul 7, 2012 at 4:35 PM, Reindl Harald wrote: > > > Am 07.07.2012 16:26, schrieb Malloc Kilobyte: > > Appreciating all Dovecot rich features, I lack just one. And this is the > > ability to customize the "quota exceeded, message rejected" message. I > know > > I can set it's default content using quota_exceeded_message parameter, > but > > i would also like to have ability to set: > > > > - to whom the message is sent. Some mail systems send copy not only to > the > > sender, but also to the recipient. It makes sense in case rejected > message > > has big size. > > > > - it's subject > > > > - if the rejected message is being attached ( with or without it's own > > attachments ) or not > > > > Moreover, I think it would make sense if the "rejected messege" contained > > information about size of rejected message and current mailbox space > usage. > > hmm - dovecot doe snot send any mail > so there is not sobject and to whom the message is sen > > messages are typically only REJECTED eith status code and status message > > the mail itself is based on this jerecht code/message from the > delivering MTA > > I don't expect Dovecot to handle the delivery. Delivery could be dispatched to MTA, if for example Dovecot let me trigger an outside script in "quota exceeded" situation, passing some information and rejected message. Similar to the way quota warnings are handled. From jfriesse at gmail.com Sun Jul 8 10:42:50 2012 From: jfriesse at gmail.com (Jan Friesse) Date: Sun, 8 Jul 2012 09:42:50 +0200 Subject: [Dovecot] quota fs: Support for hardlimit quotas (where softquota is unset) Message-ID: <20120708094250.569c762d.jfriesse@gmail.com> Hi, included is patch which adds support for calculating fs quota from hardlimit where softlimit is unset. I'm usually not setting softlimit on my server causing dovecot work incorrectly (not reporting quota at all). Regards, Jan Friesse -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot-0001-Support-for-fs-hard-limit.patch Type: application/octet-stream Size: 2956 bytes Desc: not available URL: From wojtek at wojtek.tensor.gdynia.pl Sun Jul 8 15:18:46 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sun, 8 Jul 2012 14:18:46 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF9591C.20501@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <4FF9591C.20501@thelounge.net> Message-ID: > there is more than the connection speed > > 6Gbsdo not help you much as long the physical disk can not > write in this speed and more concurretn writes making > this worser - so there are many things like big battery backed > caches fon a SAN which are imprtant for OVERALL performance with cache as big as disks itself it MAY even reach reasonable performance. This discussion is funny to read. Keep them going please! From daniel.parthey at informatik.tu-chemnitz.de Sun Jul 8 15:38:39 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 8 Jul 2012 14:38:39 +0200 Subject: [Dovecot] Dovecot 2.1.8 crashes when doing POP3 UIDL on empty INBOX In-Reply-To: References: <1734F789-EF17-43EC-8C97-E593C20FBD97@JLAssocs.com> Message-ID: <20120708123838.GA7245@daniel.localdomain> Timo Sirainen wrote: > On 8.7.2012, at 10.07, J E Lyon wrote: > > > I think some of the things Timo does, he is able to do in his sleep . . > > What's annoying is that I sometimes do, and all the hard work gets lost somewhere to the dream world :( Maybe it's just getting sorted out in your unconscious mind before you correctly type it into the real code. ;) Daniel -- https://plus.google.com/103021802792276734820 From patrickdk at patrickdk.com Sun Jul 8 16:27:27 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 08 Jul 2012 09:27:27 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> Message-ID: <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> Quoting Wojciech Puchar : >> I think there are optimal situations where any configuration looks >> good . . How often can a real-world disk actually deliver the 6Gbs >> when only a minority of disk reads are long sequential runs on the >> platters? > none of hard drives can saturate 1.5Gb/s There are many disks out that do 150-200MB/sec, easily exceeding 1.5gb/s speeds. From CMarcus at Media-Brokers.com Sun Jul 8 18:31:16 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 08 Jul 2012 11:31:16 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF9591C.20501@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <4FF9591C.20501@thelounge.net> Message-ID: <4FF9A7C4.6090508@Media-Brokers.com> On 2012-07-08 5:55 AM, Reindl Harald wrote: > nobody is using 100 MBit for a SAN And no one who is using a SAN is using 100Mb on the LAN either. In fact, I'd say that even 99.9% of all LANs - even small (wired) home LANs are Gb... -- Best regards, Charles From stan at hardwarefreak.com Mon Jul 9 00:29:09 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 08 Jul 2012 16:29:09 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> Message-ID: <4FF9FBA5.2060208@hardwarefreak.com> On 7/8/2012 8:27 AM, Patrick Domack wrote: > > Quoting Wojciech Puchar : > >>> I think there are optimal situations where any configuration looks >>> good . . How often can a real-world disk actually deliver the 6Gbs >>> when only a minority of disk reads are long sequential runs on the >>> platters? >> none of hard drives can saturate 1.5Gb/s > > There are many disks out that do 150-200MB/sec, easily exceeding 1.5gb/s > speeds. There are a few SAS drives that can saturate a 150MB/s link, such as the Seagate Cheetah 15k.7, which can sustain 204MB/s streaming read on the outer tracks. But, again, streaming rate is irrelevant to mail storage. What matters is random seek latency. And the faster the spindle, the lower the latency. Thus 15k Seagate SAS drives are excellent candidates for mail store duty, as are any 10k or 15k drives. -- Stan From wojtek at wojtek.tensor.gdynia.pl Mon Jul 9 00:36:56 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sun, 8 Jul 2012 23:36:56 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FF9FBA5.2060208@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> Message-ID: > is random seek latency. And the faster the spindle, the lower the > latency. Thus 15k Seagate SAS drives are excellent candidates for mail > store duty, as are any 10k or 15k drives. definitely not counting by $/IOPS rate. even worse looking with $/GB which is more important unless you make <1GB mail account. From patrickdk at patrickdk.com Mon Jul 9 02:24:15 2012 From: patrickdk at patrickdk.com (Patrick Domack) Date: Sun, 08 Jul 2012 19:24:15 -0400 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> Message-ID: <20120708192415.Horde._e2VbZLnE6FP_hafN9Glc9A@mail.patrickdk.com> Quoting Wojciech Puchar : >> is random seek latency. And the faster the spindle, the lower the >> latency. Thus 15k Seagate SAS drives are excellent candidates for mail >> store duty, as are any 10k or 15k drives. > definitely not counting by $/IOPS rate. even worse looking with $/GB > which is more important unless you make <1GB mail account. It largely depends what type of users your supporting. One system, I have 300 users, and it have an load on the drives of >500 iops a lot, need 8disk raid10 to support the load, about 300gigs worth of email. Another system, has 10's of k of users, storage of about 2tb and normal usage of about 7k iops. Both of these systems have a 10gig limit per user account. I find most of the time users don't max out the mailstore, unless they stop using that account. So in both of my cases, $/IOPS is much more important than $/GB, as the GB just goes to waste. From stan at hardwarefreak.com Mon Jul 9 02:59:42 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Sun, 08 Jul 2012 18:59:42 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFA06C8.20802@mirix.org> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> Message-ID: <4FFA1EEE.1050807@hardwarefreak.com> On 7/8/2012 5:16 PM, Matthias-Christian Ott wrote: > On 2012-07-08 23:29, Stan Hoeppner wrote: >> On 7/8/2012 8:27 AM, Patrick Domack wrote: >>> >>> Quoting Wojciech Puchar : >>> >>>>> I think there are optimal situations where any configuration looks >>>>> good . . How often can a real-world disk actually deliver the 6Gbs >>>>> when only a minority of disk reads are long sequential runs on the >>>>> platters? >>>> none of hard drives can saturate 1.5Gb/s >>> >>> There are many disks out that do 150-200MB/sec, easily exceeding 1.5gb/s >>> speeds. >> >> There are a few SAS drives that can saturate a 150MB/s link, such as the >> Seagate Cheetah 15k.7, which can sustain 204MB/s streaming read on the >> outer tracks. >> >> But, again, streaming rate is irrelevant to mail storage. What matters >> is random seek latency. And the faster the spindle, the lower the >> latency. Thus 15k Seagate SAS drives are excellent candidates for mail >> store duty, as are any 10k or 15k drives. > > This is simply not true. SATA SSDs (~0,66 ?/GB) are as expensive as the > 15k Seagate SAS HDDs (0,63 ?/GB), but definitely faster. First, this sub discussion in this thread has been dealing strictly with mechanical storage, and that was intentional. Now you've injected solid state storage, and in the process you first disagreed with my statement, then agreed with it. Apparently you didn't realize you did so. Would you please clarify what I stated that is "simply not true"? You comment WRT SSD doesn't prove anything I said to be untrue. Quite the contrary, you reinforced my statements. -- Stan From wojtek at wojtek.tensor.gdynia.pl Mon Jul 9 08:48:18 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 9 Jul 2012 07:48:18 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFA1EEE.1050807@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> Message-ID: > disagreed with my statement, then agreed with it. Apparently you didn't > realize you did so. Would you please clarify what I stated that is > "simply not true"? You comment WRT SSD doesn't prove anything I said to > be untrue. Quite the contrary, you reinforced my statements. > Actually the only storage i use are mainstream SATA drives and SSDs. first for size, low price and reliability, second (if needed) for speed. From h.reindl at thelounge.net Mon Jul 9 11:17:15 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 09 Jul 2012 10:17:15 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> Message-ID: <4FFA938B.1000401@thelounge.net> Am 09.07.2012 07:48, schrieb Wojciech Puchar: >> disagreed with my statement, then agreed with it. Apparently you didn't >> realize you did so. Would you please clarify what I stated that is >> "simply not true"? You comment WRT SSD doesn't prove anything I said to >> be untrue. Quite the contrary, you reinforced my statements. >> > Actually the only storage i use are mainstream SATA drives ouch - that said and your offlist discussion why SAN storages are crap for you gives a picture - nobody, really nobody is using SATA for any production-storage -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From robert at schetterer.org Mon Jul 9 11:33:07 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 09 Jul 2012 10:33:07 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFA938B.1000401@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> Message-ID: <4FFA9743.8080408@schetterer.org> Am 09.07.2012 10:17, schrieb Reindl Harald: > > > Am 09.07.2012 07:48, schrieb Wojciech Puchar: >>> disagreed with my statement, then agreed with it. Apparently you didn't >>> realize you did so. Would you please clarify what I stated that is >>> "simply not true"? You comment WRT SSD doesn't prove anything I said to >>> be untrue. Quite the contrary, you reinforced my statements. >>> >> Actually the only storage i use are mainstream SATA drives > > ouch - that said and your offlist discussion why SAN storages > are crap for you gives a picture - nobody, really nobody is > using SATA for any production-storage > Hi, sorry SATA is running fine here, by the way please show tec detail measures about filesystems , different storages, hardware envolved and tec evidence for your discusions related to dovecot. I know this is a difficult theme, and test results may controversial, but it shouldnt be a point for hot flames so it may be better, doing it offlist. Meanwhile why dont just post real working storage/filesystems examples with big used dovecot setups, which may help everybody, perhaps in the wiki After all we decide doing a big test setup to clear all this questions. -- Best Regards MfG Robert Schetterer From amateo at um.es Mon Jul 9 11:45:08 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 09 Jul 2012 10:45:08 +0200 Subject: [Dovecot] Timeout (180s) while waiting for lock for transaction log file Message-ID: <4FFA9A14.1080607@um.es> Hello, We still have problems with our updated dovecot servers... The problem now is that we are having errors like: Jul 9 10:22:02 myotis31 dovecot: lmtp(15431, ): Error: Timeout (180s) while waiting for lock for transaction log file /var/indexes//.INBOX/dovecot.index.log Jul 9 10:22:02 myotis31 dovecot: lmtp(15431, ): Error: +4YfCwKS+k9HPAAAmo+UlA: sieve: msgid=<4FFA8700.2080003 at um.es>: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-07-09 10:19:02] Jul 9 10:22:02 myotis31 dovecot: lmtp(15431, ): Error: +4YfCwKS+k9HPAAAmo+UlA: sieve: script /home/pdi/88/185188/.dovecot.sieve failed with unsuccessful implicit keep (user logfile /home/pdi/88/185188/.dovecot.sieve.log may reveal additional details) in the .dovecot.sieve.log all we have is just: sieve: info: started log at Jul 09 10:22:02. error: msgid=<4FFA8700.2080003 at um.es>: failed to store into mailbox 'INBOX': Internal error occurred. Refer to server log for more information. [2012-07-09 10:19:02]. maildir are in a nfs storage, but indexes are in local disk. I don't know if the problem could be related with a some "specific" messages or something specific to the user, because we are having this problem only with a few users and the problem repeatedly happens when our mta tries to deliver the queued message. Any idea? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 -------------- next part -------------- # 2.1.5: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-24-generic x86_64 Ubuntu 12.04 LTS auth_cache_size = 20 M auth_cache_ttl = 1 days auth_debug = yes auth_master_user_separator = * auth_verbose = yes default_process_limit = 1000 disable_plaintext_auth = no log_timestamp = %Y-%m-%d %H:%M:%S login_trusted_networks = 155.54.211.176/28 mail_debug = yes mail_location = maildir:~/Maildir:INDEX=/var/indexes/%n mail_privileged_group = mail mdbox_rotate_size = 20 M passdb { args = /etc/dovecot/master-users driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = session=yes dovecot driver = pam } plugin { lazy_expunge = .EXPUNGED/ .DELETED/ .DELETED/.EXPUNGED/ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +imapflags sieve_max_redirects = 15 zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster at um.es service anvil { client_limit = 2003 } service auth { client_limit = 3000 unix_listener auth-userdb { mode = 0666 } } service doveadm { inet_listener { port = 24245 } } service imap { process_limit = 5120 process_min_avail = 6 vsz_limit = 512 M } service lmtp { inet_listener lmtp { port = 24 } process_min_avail = 10 vsz_limit = 512 M } service pop3 { process_min_avail = 6 } ssl = no ssl_cert = } From wojtek at wojtek.tensor.gdynia.pl Mon Jul 9 12:41:46 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 9 Jul 2012 11:41:46 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFA9743.8080408@schetterer.org> References: <4FF4A12C.30704@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> Message-ID: >> using SATA for any production-storage >> > > Hi, sorry SATA is running fine here, as well here. Many people just want to be proud, or want to make things expensive so their clients are proud. but not always it's like that. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 9 12:42:30 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 9 Jul 2012 11:42:30 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFA938B.1000401@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> Message-ID: > ouch - that said and your offlist discussion why SAN storages > are crap for you gives a picture - nobody, really nobody is > using SATA for any production-storage In a view of moron you are "everyone". From role.Dovecot-Readers at JLAssocs.com Mon Jul 9 12:46:30 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Mon, 9 Jul 2012 10:46:30 +0100 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> Message-ID: <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> On 9 Jul 2012, at 10:41, Wojciech Puchar wrote: > Many people just want to be proud, or want to make things expensive so their clients are proud. but not always it's like that. You go on a bit about "pride in complexity" . . What you fail to understand is that many highly intelligent, experienced, very able engineers build systems that are as complex and as large as they _need to be_ and just because you don't deal with such large systems doesn't make everyone else wrong. (Okay, I know, some people are proud, and some people do make bad decisions about large complex systems -- but you make the mistake of assuming everyone does.) Just my 0.02 -- hope it helps. From h.reindl at thelounge.net Mon Jul 9 12:50:51 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 09 Jul 2012 11:50:51 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> Message-ID: <4FFAA97B.6080506@thelounge.net> Am 09.07.2012 11:41, schrieb Wojciech Puchar: >>> using SATA for any production-storage >>> >> >> Hi, sorry SATA is running fine here, > > as well here. > > Many people just want to be proud, or want to make things expensive so their clients are proud. but not always it's > like that. oh yeah show me the SATA drives with dual-channel http://en.wikipedia.org/wiki/Serial_Attached_SCSI Dual ports allowing redundant paths in the case of a drive-failure all your cheapiness will strike back what you do not realize that expensive hardware is NOTHING to be proud of having a business means things have to run 365 days without downtimes and as redundant as possible especially in failure cases -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From noel.butler at ausics.net Mon Jul 9 13:01:53 2012 From: noel.butler at ausics.net (Noel Butler) Date: Mon, 09 Jul 2012 20:01:53 +1000 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> References: <4FF4A12C.30704@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> Message-ID: <1341828113.10602.25.camel@tardis> Seem some people have never heard of "keep it simple, stupid" or "less is more" ... sounds like a few people here are falsely propping up their worth to their employers, making unnecessary BS to justify their own existence. My experience of over 20 years of this industry easily shows that those who try to make complex networks _always_ have far higher fail rates than those that keep it simple, nearly never have any problems, and when they do its because the powers failed, the gennies didnt kick in and UPS's died before engineers got the gennies going. I've also seen most networks that use SAN's have a far more problems than those using NAS's - providing you use decent NAS gear like EMC or Netapp :) Don't evne think about using a general server as a NAS. On Mon, 2012-07-09 at 10:46 +0100, J E Lyon wrote: > On 9 Jul 2012, at 10:41, Wojciech Puchar wrote: > > > Many people just want to be proud, or want to make things expensive so their clients are proud. but not always it's like that. > > You go on a bit about "pride in complexity" . . What you fail to understand is that many highly intelligent, experienced, very able engineers build systems that are as complex and as large as they _need to be_ and just because you don't deal with such large systems doesn't make everyone else wrong. (Okay, I know, some people are proud, and some people do make bad decisions about large complex systems -- but you make the mistake of assuming everyone does.) > > Just my 0.02 -- hope it helps. > -------------- next part -------------- A non-text attachment was scrubbed... Name: face-smile.png Type: image/png Size: 873 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From h.reindl at thelounge.net Mon Jul 9 13:10:15 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 09 Jul 2012 12:10:15 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <1341828113.10602.25.camel@tardis> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> <1341828113.10602.25.camel@tardis> Message-ID: <4FFAAE07.2000005@thelounge.net> what you do not understand is that a proper SAN is NOT an complex setup, it is in many cases a simpler one because you have TWO controllers, disks with DUAL channel and a proper RAID level in ONE device to built all this redundancy at your own is a much complexer software-setup and you can be pretty sure taht as long you are not dealing night and day with storage/Failover-setups the people designing a SAN have much more expierience what the are doing Am 09.07.2012 12:01, schrieb Noel Butler: > Seem some people have never heard of "keep it simple, stupid" or > "less is more" ... sounds like a few people here are falsely propping > up their worth to their employers, making unnecessary BS to justify > their own existence. > > My experience of over 20 years of this industry easily shows that those > who try to make complex networks _always_ have far higher fail rates > than those that keep it simple, nearly never have any problems, and when > they do its because the powers failed, the gennies didnt kick in and > UPS's died before engineers got the gennies going. I've also seen most > networks that use SAN's have a far more problems than those using NAS's > - providing you use decent NAS gear like EMC or Netapp :) Don't evne > think about using a general server as a NAS. > > > On Mon, 2012-07-09 at 10:46 +0100, J E Lyon wrote: > >> On 9 Jul 2012, at 10:41, Wojciech Puchar wrote: >> >>> Many people just want to be proud, or want to make things expensive so their clients are proud. but not always it's like that. >> >> You go on a bit about "pride in complexity" . . What you fail to understand is that many highly intelligent, experienced, very able engineers build systems that are as complex and as large as they _need to be_ and just because you don't deal with such large systems doesn't make everyone else wrong. (Okay, I know, some people are proud, and some people do make bad decisions about large complex systems -- but you make the mistake of assuming everyone does.) >> >> Just my 0.02 -- hope it helps. >> > > > -- Reindl Harald the lounge interactive design GmbH A-1060 Vienna, Hofm?hlgasse 17 CTO / CISO / Software-Development p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40 icq: 154546673, http://www.thelounge.net/ http://www.thelounge.net/signature.asc.what.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From wojtek at wojtek.tensor.gdynia.pl Mon Jul 9 13:14:57 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 9 Jul 2012 12:14:57 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> Message-ID: > >> Many people just want to be proud, or want to make things expensive so their clients are proud. but not always it's like that. > > You go on a bit about "pride in complexity" . . What you fail to understand is that many highly intelligent, experienced, very able engineers build systems that are as complex and as large as they _need to be_ and just because you don't deal with such large systems doesn't make everyone else wrong. (Okay, I know, some people are proud, and some people do make bad decisions about large complex systems -- but you make the mistake of assuming everyone does.) there are lots of skilled engineers here on that forum. But certainly not the ones that babble about storage, SANs etc.. just because they sell them, have profits from selling them or are just that uneducated. The wording they use ("everyone do this", "because it is enterprise") just proves that most of them are people that should not even touch a computer. Sadly that's majority of "specialists" nowadays. From h.reindl at thelounge.net Mon Jul 9 13:26:51 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 09 Jul 2012 12:26:51 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> Message-ID: <4FFAB1EB.5020003@thelounge.net> Am 09.07.2012 12:14, schrieb Wojciech Puchar: > there are lots of skilled engineers here on that forum. > > But certainly not the ones that babble about storage, SANs etc.. just because they sell them, have profits from > selling them or are just that uneducated. please stop this bullshit, especially OFF-LIST i do not sell them and i am not uneducated > The wording they use ("everyone do this", "because it is enterprise") just proves that most of them are people that > should not even touch a computer look in the mirror after your off-list explanations why are you re-invent the wheel and that you are not need ONLINE migration of running servers maybe you are the one who sould even not touch a computer -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From wojtek at wojtek.tensor.gdynia.pl Mon Jul 9 13:37:25 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 9 Jul 2012 12:37:25 +0200 (CEST) Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFAB1EB.5020003@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> <4FFAB1EB.5020003@thelounge.net> Message-ID: > > please stop this bullshit, especially OFF-LIST Fortunately you do not decide about it. > i do not sell them and i am not uneducated > >> The wording they use ("everyone do this", "because it is enterprise") just proves that most of them are people that >> should not even touch a computer > > look in the mirror after your off-list explanations why i am talking both about your off-list and on-list conversation in which you prove all your "knowledge" is advert based. Words like "mainstream", "enterprise", etc. are just marketing words, and should not be used in technical discussion having no meaning. Using them prooves lack of real knowledge, and that's what you exactly did. Saying to me - a regular massive user of cheap ATA/SATA drive in production since 14 years - that "nobody" use them in production is just funny! But OK - you at least plan disaster recovery. Rare event. Your clients after paying huge money to get high complexity and low performance, AT LEAST will not loose data. From h.reindl at thelounge.net Mon Jul 9 13:44:40 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 09 Jul 2012 12:44:40 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: References: <4FF4A12C.30704@gmail.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> <4FFAB1EB.5020003@thelounge.net> Message-ID: <4FFAB618.50003@thelounge.net> Am 09.07.2012 12:37, schrieb Wojciech Puchar: >> look in the mirror after your off-list explanations why > > i am talking both about your off-list and on-list conversation in which you prove all your "knowledge" is advert > based. you have no idea about my knowledge impossible becaus i have no time to explain you only a little part > Words like "mainstream", "enterprise", etc. are just marketing words, and should not be used in technical > discussion having no meaning. Using them prooves lack of real knowledge, and that's what you exactly did. how foolish can anybody be for such poor argumentation > Saying to me - a regular massive user of cheap ATA/SATA drive in production since 14 years - that "nobody" use them > in production is just funny! luck is no base for a infrastructure your life depends > But OK - you at least plan disaster recovery. Rare event. Your clients after paying huge money to get high > complexity and low performance, AT LEAST will not loose data. are you completly silly? who let's you imagine that my clients pay "huge money" because SAN? the level of complexity has not to interest any client said that: "low performance" is bullshit stop qulify things you do not know as long our webservers are generating cms-contents in around 0.011 seconds per request you are really not in the position to qualify anybodys performance as long as the following are your OFF-LIST answers you better shut up here! -------- Original-Nachricht -------- Betreff: Re: [Dovecot] Howto add another disk storage Datum: Mon, 9 Jul 2012 11:43:11 +0200 (CEST) Von: Wojciech Puchar An: Reindl Harald repeat it twice. repeat in 10 times. just for sure as maybe some of your clients will find out you are just another moron. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From r.vicinus at metaways.de Mon Jul 9 13:45:01 2012 From: r.vicinus at metaways.de (Reinhard Vicinus) Date: Mon, 09 Jul 2012 12:45:01 +0200 Subject: [Dovecot] configuring imap_stats Message-ID: <4FFAB62D.8020607@metaways.de> Hi, i'm trying to configure imap_stats like it's documented in the wiki: http://wiki2.dovecot.org/Statistics. But as soon as i activate the imap_stats plugin i get the following error message trying to login via imap: dovecot: imap: Error: Can't load plugin imap_stats_plugin: Plugin stats must be loaded also (you must set: mail_plugins=$mail_plugins stats) But the stats plugin is configured as you can see in the attached configurations. Also i can get statistics without the imap_stats enabled, but as far as i know only of the pop3 users. Kind regards Reinhard -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dovecot-with-imap-stats.txt URL: From CMarcus at Media-Brokers.com Mon Jul 9 13:47:44 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 09 Jul 2012 06:47:44 -0400 Subject: [Dovecot] Enough - Was: Re: Howto add another disk storage In-Reply-To: <4FFAB618.50003@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> <4FFAB1EB.5020003@thelounge.net> <4FFAB618.50003@thelounge.net> Message-ID: <4FFAB6D0.8090707@Media-Brokers.com> Please, guys, lets put an end to this topic, I think everyone has had their say, and everyone is capable of making up their own mind about what type of storage to use - the noise is starting to get irritating. On 2012-07-09 6:44 AM, Reindl Harald wrote: > Am 09.07.2012 12:37, schrieb Wojciech Puchar: >>> look in the mirror after your off-list explanations why >> >> i am talking both about your off-list and on-list conversation in which you prove all your "knowledge" is advert >> based. > > you have no idea about my knowledge > impossible becaus i have no time to explain you only a little part > >> Words like "mainstream", "enterprise", etc. are just marketing words, and should not be used in technical >> discussion having no meaning. Using them prooves lack of real knowledge, and that's what you exactly did. > > how foolish can anybody be for such poor argumentation > >> Saying to me - a regular massive user of cheap ATA/SATA drive in production since 14 years - that "nobody" use them >> in production is just funny! > > luck is no base for a infrastructure your life depends > >> But OK - you at least plan disaster recovery. Rare event. Your clients after paying huge money to get high >> complexity and low performance, AT LEAST will not loose data. > > are you completly silly? > > who let's you imagine that my clients pay "huge money" because SAN? > > the level of complexity has not to interest any client > said that: "low performance" is bullshit > stop qulify things you do not know > > as long our webservers are generating cms-contents in around 0.011 seconds > per request you are really not in the position to qualify anybodys > performance > > as long as the following are your OFF-LIST answers you > better shut up here! > > -------- Original-Nachricht -------- > Betreff: Re: [Dovecot] Howto add another disk storage > Datum: Mon, 9 Jul 2012 11:43:11 +0200 (CEST) > Von: Wojciech Puchar > An: Reindl Harald > > repeat it twice. repeat in 10 times. just for sure as maybe some of your > clients will find out you are just another moron. > -- Best regards, Charles Marcus I.T. Director Media Brokers International, Inc. 678.514.6200 x224 | 678.514.6299 fax From amateo at um.es Mon Jul 9 14:39:34 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 09 Jul 2012 13:39:34 +0200 Subject: [Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument Message-ID: <4FFAC2F6.4040001@um.es> Hello, More problems (it seems that we won't be able to put our new servers in production). The problem now is that when we reach 1000 dovecot processes we have errors like: Jul 9 12:10:22 myotis31 dovecot: imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument Jul 9 12:10:22 myotis31 dovecot: imap-login: Fatal: master: service(imap-login): child 32251 killed with signal 6 (core dumps disabled) and server doesn't accept any new connection. Our servers are running in ubuntu 12.04 with kernel 3.2.0-24-generic kernel. I have found a similar problem at https://bugzilla.redhat.com/show_bug.cgi?id=681578 but it is for redhat kernel of series 2.6. Could be the same problem? Any pointer to this problem for ubuntu kernels? Or for generic 3.0 kernels? Any idea? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From tss at iki.fi Mon Jul 9 14:51:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 09 Jul 2012 14:51:26 +0300 Subject: [Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument In-Reply-To: <4FFAC2F6.4040001@um.es> References: <4FFAC2F6.4040001@um.es> Message-ID: <1341834686.19913.27.camel@hurina> On Mon, 2012-07-09 at 13:39 +0200, Angel L. Mateo wrote: > Hello, > > More problems (it seems that we won't be able to put our new servers in > production). The problem now is that when we reach 1000 dovecot > processes we have errors like: > > Jul 9 12:10:22 myotis31 dovecot: imap-login: Panic: epoll_ctl(add, 6) > failed: Invalid argument > Jul 9 12:10:22 myotis31 dovecot: imap-login: Fatal: master: > service(imap-login): child 32251 killed with signal 6 (core dumps disabled) > > and server doesn't accept any new connection. > > Our servers are running in ubuntu 12.04 with kernel 3.2.0-24-generic > kernel. I have found a similar problem at > https://bugzilla.redhat.com/show_bug.cgi?id=681578 but it is for redhat > kernel of series 2.6. > > Could be the same problem? It's definitely the same problem. > Any pointer to this problem for ubuntu > kernels? Or for generic 3.0 kernels? Sorry, no idea. From amateo at um.es Mon Jul 9 15:03:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Mon, 09 Jul 2012 14:03:42 +0200 Subject: [Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument In-Reply-To: <1341834686.19913.27.camel@hurina> References: <4FFAC2F6.4040001@um.es> <1341834686.19913.27.camel@hurina> Message-ID: <4FFAC89E.1020805@um.es> El 09/07/12 13:51, Timo Sirainen escribi?: > On Mon, 2012-07-09 at 13:39 +0200, Angel L. Mateo wrote: >> Hello, >> >> More problems (it seems that we won't be able to put our new servers in >> production). The problem now is that when we reach 1000 dovecot >> processes we have errors like: >> >> Jul 9 12:10:22 myotis31 dovecot: imap-login: Panic: epoll_ctl(add, 6) >> failed: Invalid argument >> Jul 9 12:10:22 myotis31 dovecot: imap-login: Fatal: master: >> service(imap-login): child 32251 killed with signal 6 (core dumps disabled) >> >> and server doesn't accept any new connection. >> >> Our servers are running in ubuntu 12.04 with kernel 3.2.0-24-generic >> kernel. I have found a similar problem at >> https://bugzilla.redhat.com/show_bug.cgi?id=681578 but it is for redhat >> kernel of series 2.6. >> >> Could be the same problem? > > It's definitely the same problem. > The patch mentioned in that thread to solve the problem (https://bugzilla.redhat.com/attachment.cgi?id=570690&action=diff#a/fs/eventpoll.c_sec1) consist on adding: static int path_count_inc(int nests) { if (nests == 0) return 0; to the begining of path_count_inc function at fs/eventpoll.c in kernel source. I have checked that 3.2.0 kernel I have and this patch is already applied. Any other idea? Is there anybody running dovecot with a ubuntu/debian/vanilla (not redhat based) kernel? What kernel? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From robert at schetterer.org Mon Jul 9 15:37:41 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 09 Jul 2012 14:37:41 +0200 Subject: [Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument In-Reply-To: <1341834686.19913.27.camel@hurina> References: <4FFAC2F6.4040001@um.es> <1341834686.19913.27.camel@hurina> Message-ID: <4FFAD095.4050804@schetterer.org> Am 09.07.2012 13:51, schrieb Timo Sirainen: > On Mon, 2012-07-09 at 13:39 +0200, Angel L. Mateo wrote: >> Hello, >> >> More problems (it seems that we won't be able to put our new servers in >> production). The problem now is that when we reach 1000 dovecot >> processes we have errors like: >> >> Jul 9 12:10:22 myotis31 dovecot: imap-login: Panic: epoll_ctl(add, 6) >> failed: Invalid argument >> Jul 9 12:10:22 myotis31 dovecot: imap-login: Fatal: master: >> service(imap-login): child 32251 killed with signal 6 (core dumps disabled) >> >> and server doesn't accept any new connection. >> >> Our servers are running in ubuntu 12.04 with kernel 3.2.0-24-generic >> kernel. I have found a similar problem at >> https://bugzilla.redhat.com/show_bug.cgi?id=681578 but it is for redhat >> kernel of series 2.6. >> >> Could be the same problem? > > It's definitely the same problem. > >> Any pointer to this problem for ubuntu >> kernels? Or for generic 3.0 kernels? > > Sorry, no idea. > > in case its fixed in some more recent versions http://kernel.ubuntu.com/~kernel-ppa/mainline/ guess this is latest for 12.04 http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/ -- Best Regards MfG Robert Schetterer From micah at riseup.net Mon Jul 9 18:01:49 2012 From: micah at riseup.net (Micah Anderson) Date: Mon, 09 Jul 2012 11:01:49 -0400 Subject: [Dovecot] mailboxes missing after upgrade from 2.1.4 to 2.1.7 References: <87txxqxmsq.fsf@algae.riseup.net> Message-ID: <87txxh2d2a.fsf@algae.riseup.net> Micah Anderson writes: > I mentioned this on the #dovecot irc channel, but I thought I would post > here so I can provide more details. > > The basic problem is that when I upgraded from 2.1.4 to 2.1.7, some > users no longer are able to see their folders in pine/alpine. The > folders are actually there, and are subscribed (according to doveadm) > and its possible to do operations on those folders, if you know the > name (for example, opening the folder works fine if you know the name). > > When I downgrade to 2.1.4 again, the folders appear again, so its > clearly something that changed between the two versions. I looked > through the changelog, and found a few possibile culprits, I think > pine/alpine use imapc, so that could be related: I was able to solve this by changing the alpine folder-collection configuration to *not* have /INBOX.[] at the end and instead have just []! micah From user+dovecot at localhost.localdomain.org Mon Jul 9 21:25:38 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 09 Jul 2012 20:25:38 +0200 Subject: [Dovecot] configuring imap_stats In-Reply-To: <4FFAB62D.8020607@metaways.de> References: <4FFAB62D.8020607@metaways.de> Message-ID: <4FFB2222.5010001@localhost.localdomain.org> On 07/09/2012 12:45 PM Reinhard Vicinus wrote: > Hi, > > i'm trying to configure imap_stats like it's documented in the wiki: > http://wiki2.dovecot.org/Statistics. But as soon as i activate the > imap_stats plugin i get the following error message trying to login via > imap: > > dovecot: imap: Error: Can't load plugin imap_stats_plugin: Plugin stats > must be loaded also (you must set: mail_plugins=$mail_plugins stats) The global mail_plugins setting should contain: mail_plugins = quota stats And "protocol imap"'s mail_plugins setting should contain: mail_plugins = $mail_plugins imap_quota imap_stats The $mail_plugins inside protocol imap {} will be expanded to the value of your global mail_plugins setting. ,--[ doveconf -f protocol=imap mail_plugins ]-- | mail_plugins = quota stats imap_quota imap_stats `-- Regards, Pascal -- The trapper recommends today: c01dcofe.1219120 at localdomain.org From stan at hardwarefreak.com Mon Jul 9 22:29:19 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 09 Jul 2012 14:29:19 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFA938B.1000401@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> Message-ID: <4FFB310F.5000000@hardwarefreak.com> On 7/9/2012 3:17 AM, Reindl Harald wrote: > > > Am 09.07.2012 07:48, schrieb Wojciech Puchar: >>> disagreed with my statement, then agreed with it. Apparently you didn't >>> realize you did so. Would you please clarify what I stated that is >>> "simply not true"? You comment WRT SSD doesn't prove anything I said to >>> be untrue. Quite the contrary, you reinforced my statements. >>> >> Actually the only storage i use are mainstream SATA drives > > ouch - that said and your offlist discussion why SAN storages > are crap for you gives a picture - nobody, really nobody is > using SATA for any production-storage That's simply not true Reindl. SATA drives are being used very widely in production today, and outnumber SAS deployments by a very wide margin. For instance, web servers, counting in the 10s of millions worldwide, use SATA drives exclusively, where RAM is critical for performance, not disk. You won't find SAS drives in web farms simply due to cost. Everyone doing D2D backup and nearline storage is using SATA today. And most everyone, from large enterprises, to K-12 schools, to SOHOs, use SATA drives in front line file servers. SAS is found today almost exclusively in high volume transactional servers such as mail spools, mail stores, databases, VM image storage, and applications that need higher reliability, such as medical imaging systems, etc. -- Stan From h.reindl at thelounge.net Mon Jul 9 22:41:25 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Mon, 09 Jul 2012 21:41:25 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFB310F.5000000@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> Message-ID: <4FFB33E5.2010408@thelounge.net> Am 09.07.2012 21:29, schrieb Stan Hoeppner: > On 7/9/2012 3:17 AM, Reindl Harald wrote: >> >> >> Am 09.07.2012 07:48, schrieb Wojciech Puchar: >>>> disagreed with my statement, then agreed with it. Apparently you didn't >>>> realize you did so. Would you please clarify what I stated that is >>>> "simply not true"? You comment WRT SSD doesn't prove anything I said to >>>> be untrue. Quite the contrary, you reinforced my statements. >>>> >>> Actually the only storage i use are mainstream SATA drives >> >> ouch - that said and your offlist discussion why SAN storages >> are crap for you gives a picture - nobody, really nobody is >> using SATA for any production-storage > > That's simply not true Reindl. > > SATA drives are being used very widely in production today, and > outnumber SAS deployments by a very wide margin. for SOHO with no public services, yes > SAS is found today almost exclusively in high volume transactional > servers such as mail spools, mail stores, databases, VM image storage, > and applications that need higher reliability, such as medical imaging > systems, etc well, we are speaking about mail spools and mail storages here i thought additionally my mail storage lives on virtual machines as any other services started some years ago and finsihed 2010 moving the last bare-metal server to VM infrastructzre in these environments you find near to zero SATA only few people these does are doing bare metal installs in days where hardware supported virtaliziation has nearly zero overhead -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From florin at andrei.myip.org Mon Jul 9 23:28:10 2012 From: florin at andrei.myip.org (Florin Andrei) Date: Mon, 09 Jul 2012 13:28:10 -0700 Subject: [Dovecot] recover .sieve plaintext from .sievec binary file Message-ID: <4FFB3EDA.301@andrei.myip.org> Hi, I upgraded a server from an old Linux distribution to a much newer one. In the process, Dovecot got upgraded from v1 to v2. By mistake, I deleted the .sieve plaintext file with all the Sieve filters, but I still do have the .sievec compiled binary. Is there any way to dump the binary file back into plaintext format? Keep in mind, I'm running v2 now, and the binary was made on a v1 Dovecot server. Thanks. -- Florin Andrei http://florin.myip.org/ From solo at privat.dk Mon Jul 9 23:42:20 2012 From: solo at privat.dk (solo at privat.dk) Date: Mon, 9 Jul 2012 22:42:20 +0200 Subject: [Dovecot] Howto add another disk storage References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> Message-ID: <18.BF.19242.C224BFF4@fep44.mail.dk> Moi. Hi there. Wouldn't it be possible to either stop this madness of silly people trying to teach other maillist users this storage nonsense ? (Religion) or to tell how to unsubscribe asap ? My inbox is filling up with this to me, and maybe other on the dovecot list, completely out of the list scope nonsense. Regards Solo ----- Original meddelelse ----- > Fra: Reindl Harald > Til: dovecot at dovecot.org > Dato: Man, 09. jul 2012 21:41 > Emne: Re: [Dovecot] Howto add another disk storage > > > > Am 09.07.2012 21:29, schrieb Stan Hoeppner: > > On 7/9/2012 3:17 AM, Reindl Harald wrote: > >> > >> > >> Am 09.07.2012 07:48, schrieb Wojciech Puchar: > >>>> disagreed with my statement, then agreed with it. Apparently you > didn't > >>>> realize you did so. Would you please clarify what I stated that > is > >>>> "simply not true"? You comment WRT SSD doesn't prove anything I > said to > >>>> be untrue. Quite the contrary, you reinforced my statements. > >>>> > >>> Actually the only storage i use are mainstream SATA drives > >> > >> ouch - that said and your offlist discussion why SAN storages > >> are crap for you gives a picture - nobody, really nobody is > >> using SATA for any production-storage > > > > That's simply not true Reindl. > > > > SATA drives are being used very widely in production today, and > > outnumber SAS deployments by a very wide margin. > > for SOHO with no public services, yes > > > SAS is found today almost exclusively in high volume transactional > > servers such as mail spools, mail stores, databases, VM image > storage, > > and applications that need higher reliability, such as medical > imaging > > systems, etc > > well, we are speaking about mail spools and mail storages here i > thought > > additionally my mail storage lives on virtual machines as any other > services > started some years ago and finsihed 2010 moving the last bare-metal > server to > VM infrastructzre > > in these environments you find near to zero SATA > only few people these does are doing bare metal installs in days > where hardware supported virtaliziation has nearly zero overhead From user+dovecot at localhost.localdomain.org Mon Jul 9 23:46:12 2012 From: user+dovecot at localhost.localdomain.org (Pascal Volk) Date: Mon, 09 Jul 2012 22:46:12 +0200 Subject: [Dovecot] recover .sieve plaintext from .sievec binary file In-Reply-To: <4FFB3EDA.301@andrei.myip.org> References: <4FFB3EDA.301@andrei.myip.org> Message-ID: <4FFB4314.3010200@localhost.localdomain.org> On 07/09/2012 10:28 PM Florin Andrei wrote: > ? > By mistake, I deleted the .sieve plaintext file with all the Sieve > filters, but I still do have the .sievec compiled binary. Is there any > way to dump the binary file back into plaintext format? Keep in mind, > I'm running v2 now, and the binary was made on a v1 Dovecot server. You could try to recover your scripts using sieve-dump(1): http://pigeonhole.dovecot.org/doc/man1/sieve-dump.1.html Regards, Pascal -- The trapper recommends today: fabaceae.1219122 at localdomain.org From florin at andrei.myip.org Mon Jul 9 23:53:23 2012 From: florin at andrei.myip.org (Florin Andrei) Date: Mon, 09 Jul 2012 13:53:23 -0700 Subject: [Dovecot] recover .sieve plaintext from .sievec binary file In-Reply-To: <4FFB4314.3010200@localhost.localdomain.org> References: <4FFB3EDA.301@andrei.myip.org> <4FFB4314.3010200@localhost.localdomain.org> Message-ID: <4FFB44C3.80605@andrei.myip.org> On 07/09/2012 01:46 PM, Pascal Volk wrote: > On 07/09/2012 10:28 PM Florin Andrei wrote: >> ? >> By mistake, I deleted the .sieve plaintext file with all the Sieve >> filters, but I still do have the .sievec compiled binary. Is there any >> way to dump the binary file back into plaintext format? Keep in mind, >> I'm running v2 now, and the binary was made on a v1 Dovecot server. > > You could try to recover your scripts using sieve-dump(1): > http://pigeonhole.dovecot.org/doc/man1/sieve-dump.1.html $ sieve-dump .dovecot.sievec sieve-dump(florin): Error: sieve: binary_open: binary .dovecot.sievec has corrupted header (0x53727943) or it is not a Sieve binary sieve-dump(florin): Error: failed to load binary: .dovecot.sievec -- Florin Andrei http://florin.myip.org/ From jc at info-systems.de Tue Jul 10 00:00:06 2012 From: jc at info-systems.de (Jakob Curdes) Date: Mon, 09 Jul 2012 23:00:06 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <18.BF.19242.C224BFF4@fep44.mail.dk> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> <18.BF.19242.C224BFF4@fep44.mail.dk> Message-ID: <4FFB4656.3010004@info-systems.de> Am 09.07.2012 22:42, schrieb solo at privat.dk: > Moi. > Hi there. > > Wouldn't it be possible to either stop this madness of silly people > trying to teach other maillist users this storage nonsense ? (Religion) > or to tell how to unsubscribe asap ? > > My inbox is filling up with this to me, and maybe other on the dovecot > list, completely out of the list scope nonsense. > Agreed. Please continue the SAS-SATA fight and the SAN-NAS fight somewhere else. Cheers JC From stephan at rename-it.nl Tue Jul 10 01:00:05 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 10 Jul 2012 00:00:05 +0200 Subject: [Dovecot] recover .sieve plaintext from .sievec binary file In-Reply-To: <4FFB44C3.80605@andrei.myip.org> References: <4FFB3EDA.301@andrei.myip.org> <4FFB4314.3010200@localhost.localdomain.org> <4FFB44C3.80605@andrei.myip.org> Message-ID: <4FFB5465.2050901@rename-it.nl> On 7/9/2012 10:53 PM, Florin Andrei wrote: > On 07/09/2012 01:46 PM, Pascal Volk wrote: >> On 07/09/2012 10:28 PM Florin Andrei wrote: >>> ? >>> By mistake, I deleted the .sieve plaintext file with all the Sieve >>> filters, but I still do have the .sievec compiled binary. Is there any >>> way to dump the binary file back into plaintext format? Keep in mind, >>> I'm running v2 now, and the binary was made on a v1 Dovecot server. >> >> You could try to recover your scripts using sieve-dump(1): >> http://pigeonhole.dovecot.org/doc/man1/sieve-dump.1.html > > $ sieve-dump .dovecot.sievec > sieve-dump(florin): Error: sieve: binary_open: binary .dovecot.sievec > has corrupted header (0x53727943) or it is not a Sieve binary > sieve-dump(florin): Error: failed to load binary: .dovecot.sievec That is not going to work. That binary is produced by the old cmusieve plugin. You'll need the old sieved tool to dump the binary. And then you still have a problem. The sieved tool does not reconstruct the sieve script that was used to create it; it produces an assembly-like output from the byte code. You'd have to manually translate that back to Sieve. :/ Regards, Stephan. From florin at andrei.myip.org Tue Jul 10 01:36:55 2012 From: florin at andrei.myip.org (Florin Andrei) Date: Mon, 09 Jul 2012 15:36:55 -0700 Subject: [Dovecot] recover .sieve plaintext from .sievec binary file In-Reply-To: <4FFB5465.2050901@rename-it.nl> References: <4FFB3EDA.301@andrei.myip.org> <4FFB4314.3010200@localhost.localdomain.org> <4FFB44C3.80605@andrei.myip.org> <4FFB5465.2050901@rename-it.nl> Message-ID: <4FFB5D07.7090406@andrei.myip.org> On 07/09/2012 03:00 PM, Stephan Bosch wrote: > > And then you still have a problem. The sieved tool does not reconstruct > the sieve script that was used to create it; it produces an > assembly-like output from the byte code. You'd have to manually > translate that back to Sieve. :/ Looks like I'll have to rebuild that file by hand. :( -- Florin Andrei http://florin.myip.org/ From stan at hardwarefreak.com Tue Jul 10 06:59:48 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 09 Jul 2012 22:59:48 -0500 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFB33E5.2010408@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> Message-ID: <4FFBA8B4.9000800@hardwarefreak.com> On 7/9/2012 2:41 PM, Reindl Harald wrote: > > > Am 09.07.2012 21:29, schrieb Stan Hoeppner: >> On 7/9/2012 3:17 AM, Reindl Harald wrote: >>> >>> >>> Am 09.07.2012 07:48, schrieb Wojciech Puchar: >>>>> disagreed with my statement, then agreed with it. Apparently you didn't >>>>> realize you did so. Would you please clarify what I stated that is >>>>> "simply not true"? You comment WRT SSD doesn't prove anything I said to >>>>> be untrue. Quite the contrary, you reinforced my statements. >>>>> >>>> Actually the only storage i use are mainstream SATA drives >>> >>> ouch - that said and your offlist discussion why SAN storages >>> are crap for you gives a picture - nobody, really nobody is >>> using SATA for any production-storage >> >> That's simply not true Reindl. >> >> SATA drives are being used very widely in production today, and >> outnumber SAS deployments by a very wide margin. > > for SOHO with no public services, yes Google has more public facing services, servers, than anyone, and Google uses only SATA drives, zero SAS. They also host more mail spools and mailboxes than anyone. Again, all on SATA drives. Now I'm sure you'll tell us why Google doesn't count for some reason in this discussion, why their use of SATA isn't relevant somehow. -- Stan From noel.butler at ausics.net Tue Jul 10 07:42:11 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 10 Jul 2012 14:42:11 +1000 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFAAE07.2000005@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFA9743.8080408@schetterer.org> <0E0A18BA-D9AE-4315-80FB-92C059D40C33@JLAssocs.com> <1341828113.10602.25.camel@tardis> <4FFAAE07.2000005@thelounge.net> Message-ID: <1341895331.7004.3.camel@tardis> On Mon, 2012-07-09 at 12:10 +0200, Reindl Harald wrote: > what you do not understand is that a proper SAN is NOT > an complex setup, it is in many cases a simpler one > because you have TWO controllers, disks with DUAL channel > and a proper RAID level in ONE device > > to built all this redundancy at your own is a much > complexer software-setup and you can be pretty sure > taht as long you are not dealing night and day with > storage/Failover-setups the people designing a > SAN have much more expierience what the are doing > Is not my experience with some pretty large ISP's, SAN might have its place for web, but not in any mail world I'm responsible for, stress testing has shown me no SAN speed difference of NAS for mail (yes this may differ is an office world, but not when dealing with the size of emails) and I sure as hell wont pay the extra gazillion these vendors want for the SAN gear. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From robert at schetterer.org Tue Jul 10 09:09:07 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Jul 2012 08:09:07 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFB310F.5000000@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF4B10B.9070906@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> Message-ID: <4FFBC703.2020705@schetterer.org> Am 09.07.2012 21:29, schrieb Stan Hoeppner: > SAS is found today almost exclusively in high volume transactional > servers such as mail spools, mail stores, databases, VM image storage, > and applications that need higher reliability, such as medical imaging > systems, etc. and SAS may not be faster always, it deeply depends on motherboard ,bios versions, drivers, date of release, storage modell etc SAS could be faster but it must not everytime and everywhere IMAP and specially mailserver storages may have other needs as other servertypes, so it depends on general parameters which you like to goal, parameters i.e may your budget number of mailboxes, size of mailboxes, number of parallel users, mailbox type, cluster filesystem ,loadbalancing setup soft/hardware raid/hardware, use drbd and/or san/nas specialized hardware etc so i found very interesting info in this thread , but at the end it is not very helpfull to decide what to use in real world mailsetups this is always a local decision by local circumstances there are no general easy answers for this theme, that fit simply all and as in real life , dont think what fits best to you , ist the best for others -- Best Regards MfG Robert Schetterer From robert at schetterer.org Tue Jul 10 09:13:29 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Jul 2012 08:13:29 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFB33E5.2010408@thelounge.net> References: <4FF4A12C.30704@gmail.com> <4FF51A14.70102@hardwarefreak.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> Message-ID: <4FFBC809.8060006@schetterer.org> Am 09.07.2012 21:41, schrieb Reindl Harald: > in these environments you find near to zero SATA > only few people these does are doing bare metal installs in days > where hardware supported virtaliziation has nearly zero overhead Hi Harald, that simply not true i have thousends of mailbox users on sata stores since years ,without any problem, you should learn that things that might fit at your place, are not the ultimate answer to everything -- Best Regards MfG Robert Schetterer From f.bonnet at esiee.fr Tue Jul 10 09:31:36 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Tue, 10 Jul 2012 08:31:36 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFBC809.8060006@schetterer.org> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> <4FFBC809.8060006@schetterer.org> Message-ID: <4FFBCC48.60309@esiee.fr> Le 10/07/2012 08:13, Robert Schetterer a ?crit : > Am 09.07.2012 21:41, schrieb Reindl Harald: >> in these environments you find near to zero SATA >> only few people these does are doing bare metal installs in days >> where hardware supported virtaliziation has nearly zero overhead > Hi Harald, that simply not true > i have thousends of mailbox users on sata stores > since years ,without any problem, you should learn that > things that might fit at your place, are not the ultimate > answer to everything > Would it be possible to close this thread from Dovecot mailing-list ? Please setup mailing-lists pros-cons-Reindl or pros-cons-sata instead Thank you very much From h.reindl at thelounge.net Tue Jul 10 09:46:27 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 10 Jul 2012 08:46:27 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFBA8B4.9000800@hardwarefreak.com> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> <4FFBA8B4.9000800@hardwarefreak.com> Message-ID: <4FFBCFC3.5030808@thelounge.net> Am 10.07.2012 05:59, schrieb Stan Hoeppner: >>> That's simply not true Reindl. >>> >>> SATA drives are being used very widely in production today, and >>> outnumber SAS deployments by a very wide margin. >> >> for SOHO with no public services, yes > > Google has more public facing services, servers, than anyone, and Google > uses only SATA drives, zero SAS. They also host more mail spools and > mailboxes than anyone. Again, all on SATA drives. > > Now I'm sure you'll tell us why Google doesn't count for some reason in > this discussion, why their use of SATA isn't relevant somehow DAMNED: why do you restart this thread again after enough mails it shopuld be stopped? finally it does not botehr me which hardware-crap who is using as long mine critical one are only 14 SAS disks for a lot of virtual servers, all with dual channel, redundant and hotplug-able ________________________________ however: because google generally has the strategy to use cheap hardware in huge masses with very high redundancy while SAS stands for high ability (dual channel ports) with higher pricses but less hardware at all -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From f.bianchi at arte.unipi.it Tue Jul 10 10:24:43 2012 From: f.bianchi at arte.unipi.it (Federico Bianchi) Date: Tue, 10 Jul 2012 09:24:43 +0200 (CEST) Subject: [Dovecot] different mail_max_userip settings Message-ID: Is it possible to have mail_max_userip set to a value for localhost (webmail) and to another value for everything else? Best regards Federico Bianchi Dipartimento di Storia delle Arti Universita` di Pisa via Trieste, 38 - I-56126 Pisa (Italy) tel.(+39) 050 221 6 024; fax (+39) 050 221 6 001 e-mail: =================================================== !DISCLAIMER!: my e-mail reflects _my_own_ opinions! =================================================== From tss at iki.fi Tue Jul 10 10:53:04 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 10 Jul 2012 09:53:04 +0200 Subject: [Dovecot] Howto add another disk storage In-Reply-To: <4FFBCC48.60309@esiee.fr> References: <4FF4A12C.30704@gmail.com> <4FF55FCD.6070001@hardwarefreak.com> <4FF663DA.9090504@hardwarefreak.com> <4FF7D92B.6070401@hardwarefreak.com> <4FF80302.2020800@thelounge.net> <20120708032755.74728b69@mydesk> <20120708033654.0709ba04@mydesk> <7F03B403-674C-4C79-B1E8-8F9039B5F49F@JLAssocs.com> <20120708092727.Horde.tEHyJJLnE6FP_Yq-rjeyRhA@mail.patrickdk.com> <4FF9FBA5.2060208@hardwarefreak.com> <4FFA06C8.20802@mirix.org> <4FFA1EEE.1050807@hardwarefreak.com> <4FFA938B.1000401@thelounge.net> <4FFB310F.5000000@hardwarefreak.com> <4FFB33E5.2010408@thelounge.net> <4FFBC809.8060006@schetterer.org> <4FFBCC48.60309@esiee.fr> Message-ID: <83578850-6E6B-4A5A-8569-1CEEF53CDF6C@iki.fi> On 10.7.2012, at 8.31, Frank Bonnet wrote: > Would it be possible to close this thread from Dovecot mailing-list ? Yeah, enough with this thread. From stan at hardwarefreak.com Tue Jul 10 11:15:06 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 10 Jul 2012 03:15:06 -0500 Subject: [Dovecot] different mail_max_userip settings In-Reply-To: References: Message-ID: <4FFBE48A.4060509@hardwarefreak.com> On 7/10/2012 2:24 AM, Federico Bianchi wrote: > > Is it possible to have mail_max_userip set to a value for localhost > (webmail) and to another value for everything else? mail_max_userip as the name suggests is a per user IMAP socket connection limit. So you should be able to set it to one value that satisfies the requirements of both the webmail app and your various MUAs. For instance, Thunderbird opens 5 connections by default last I checked. Say your webmail app or IMAP proxy between webmail and Dovecot makes one connection per user. Setting mail_max_userip to 6 should allow a user to simultaneously connect from both TBird and webmail without causing problems. Is this the type of problem you're trying to solve, or is it not this but something related? -- Stan From f.bianchi at arte.unipi.it Tue Jul 10 11:52:52 2012 From: f.bianchi at arte.unipi.it (Federico Bianchi) Date: Tue, 10 Jul 2012 10:52:52 +0200 (CEST) Subject: [Dovecot] different mail_max_userip settings In-Reply-To: <4FFBE48A.4060509@hardwarefreak.com> References: <4FFBE48A.4060509@hardwarefreak.com> Message-ID: On Tue, 10 Jul 2012, Stan Hoeppner wrote: > On 7/10/2012 2:24 AM, Federico Bianchi wrote: >> >> Is it possible to have mail_max_userip set to a value for localhost >> (webmail) and to another value for everything else? > > mail_max_userip as the name suggests is a per user IMAP socket > connection limit. So you should be able to set it to one value that > satisfies the requirements of both the webmail app and your various MUAs. > > For instance, Thunderbird opens 5 connections by default last I checked. > Say your webmail app or IMAP proxy between webmail and Dovecot makes > one connection per user. Setting mail_max_userip to 6 should allow a > user to simultaneously connect from both TBird and webmail without > causing problems. > > Is this the type of problem you're trying to solve, or is it not this > but something related? Something like that: we have users connecting from desktop PCs in the office, mobile and webmail (from xDSL IPs at home) simultaneously. Of course, they should close Thunderbird (or Apple Mail, or whatever) at work when they go home - but your solution might be OK in that respect as well. Best regards Federico Bianchi Dipartimento di Storia delle Arti Universita` di Pisa via Trieste, 38 - I-56126 Pisa (Italy) tel.(+39) 050 221 6 024; fax (+39) 050 221 6 001 e-mail: =================================================== !DISCLAIMER!: my e-mail reflects _my_own_ opinions! =================================================== From h.reindl at thelounge.net Tue Jul 10 14:00:08 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 10 Jul 2012 13:00:08 +0200 Subject: [Dovecot] dovecot 2.1.8 still tries to touch any mountpoint Message-ID: <4FFC0B38.9070603@thelounge.net> why does dovecot touch any ever used fuse-mountpoint and scream at the next start that is no longer mounted? "If this is intentional, remove it with doveadm mount" why if i never called "doveadm" on any setup at all? Jul 10 09:10:56 rh dovecot: master: Dovecot v2.1.8 starting up (core dumps disabled) Jul 10 09:10:56 rh dovecot: master: Warning: /sys/fs/cgroup/cpu is no longer mounted. If this is intentional, remove it with doveadm mount Jul 10 09:10:56 rh dovecot: master: Warning: /sys/fs/cgroup/cpuacct is no longer mounted. If this is intentional, remove it with doveadm mount Jul 10 09:10:56 rh dovecot: master: Warning: /var/run/vmblock-fuse is no longer mounted. If this is intentional, remove it with doveadm mount Jul 10 09:10:56 rh dovecot: master: Warning: /mnt/arrakis is no longer mounted. If this is intentional, remove it with doveadm mount Jul 10 09:10:56 rh dovecot: master: Warning: /mnt/loungebackup is no longer mounted. If this is intentional, remove it with doveadm mount Jul 10 09:10:56 rh dovecot: master: Warning: /mnt/panel is no longer mounted. If this is intentional, remove it with doveadm mount Jul 10 09:10:56 rh dovecot: master: Warning: /mnt/proxy is no longer mounted. If this is intentional, remove it with doveadm mount -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From voytek at sbt.net.au Tue Jul 10 14:25:55 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Tue, 10 Jul 2012 21:25:55 +1000 Subject: [Dovecot] getting rid of old spam from +spam Maildir ? Message-ID: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> I'm trying to setup per user '+spam' delivery from amavis tags, so that each user gets any mails tagged as spam to 'spam' Maildir via +spam Dovecot lmtp delivery. after say 7 days I want to delete all spams older than 7 days, if I simply delete mail files from the file system, is that a 'bad thing' ? what is a proper way to do that, and scripts ? for removing/archiving old uncollected emails in main INBOX I use archivemail from sf, but, don't think I can do subfoldrs with archivemail any pointers (and scripts) welcomed, thanks -- Voytek From wojtek at wojtek.tensor.gdynia.pl Tue Jul 10 14:27:48 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 10 Jul 2012 13:27:48 +0200 (CEST) Subject: [Dovecot] dovecot 2.1.8 still tries to touch any mountpoint In-Reply-To: <4FFC0B38.9070603@thelounge.net> References: <4FFC0B38.9070603@thelounge.net> Message-ID: > why does dovecot touch any ever used fuse-mountpoint > and scream at the next start that is no longer mounted? > > "If this is intentional, remove it with doveadm mount" > why if i never called "doveadm" on any setup at all? Because you never read any docs. From robert at schetterer.org Tue Jul 10 14:34:24 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Jul 2012 13:34:24 +0200 Subject: [Dovecot] getting rid of old spam from +spam Maildir ? In-Reply-To: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> Message-ID: <4FFC1340.6060607@schetterer.org> Am 10.07.2012 13:25, schrieb Voytek Eymont: > I'm trying to setup per user '+spam' delivery from amavis tags, so that > each user gets any mails tagged as spam to 'spam' Maildir via +spam > Dovecot lmtp delivery. i have no amavis, i use a global sieve rule to deliver tagged mail into special Junk folder, and virtual plugin to get users the chance to download tagged mails from Junk folder via pop3 too look i.e here http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples > > after say 7 days I want to delete all spams older than 7 days, > if I simply delete mail files from the file system, is that a 'bad thing' ? perhaps use daily cron with something like this doveadm expunge -A mailbox Junk savedbefore 7d; > > what is a proper way to do that, and scripts ? > > for removing/archiving old uncollected emails in main INBOX I use > archivemail from sf, but, don't think I can do subfoldrs with archivemail > > any pointers (and scripts) welcomed, thanks > > i am not sure what you ask for here, looking for archive solution ? -- Best Regards MfG Robert Schetterer From robert at schetterer.org Tue Jul 10 14:37:59 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 10 Jul 2012 13:37:59 +0200 Subject: [Dovecot] getting rid of old spam from +spam Maildir ? In-Reply-To: <4FFC1340.6060607@schetterer.org> References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> <4FFC1340.6060607@schetterer.org> Message-ID: <4FFC1417.4020004@schetterer.org> Am 10.07.2012 13:34, schrieb Robert Schetterer: > Am 10.07.2012 13:25, schrieb Voytek Eymont: >> I'm trying to setup per user '+spam' delivery from amavis tags, so that >> each user gets any mails tagged as spam to 'spam' Maildir via +spam >> Dovecot lmtp delivery. > > i have no amavis, i use a global sieve rule to deliver tagged mail into > special Junk folder, and virtual plugin to get users the chance to > download tagged mails from Junk folder via pop3 too > > look i.e here > > http://wiki2.dovecot.org/Pigeonhole/Sieve/Examples > >> >> after say 7 days I want to delete all spams older than 7 days, >> if I simply delete mail files from the file system, is that a 'bad thing' ? > > perhaps use daily cron with something like this > > doveadm expunge -A mailbox Junk savedbefore 7d; i forgot read http://wiki2.dovecot.org/Plugins/Expire for this too > >> >> what is a proper way to do that, and scripts ? >> >> for removing/archiving old uncollected emails in main INBOX I use >> archivemail from sf, but, don't think I can do subfoldrs with archivemail >> >> any pointers (and scripts) welcomed, thanks >> >> > > i am not sure what you ask for here, looking for archive solution ? > -- Best Regards MfG Robert Schetterer From wojtek at wojtek.tensor.gdynia.pl Tue Jul 10 15:10:33 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 10 Jul 2012 14:10:33 +0200 (CEST) Subject: [Dovecot] getting rid of old spam from +spam Maildir ? In-Reply-To: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> Message-ID: > Dovecot lmtp delivery. > > after say 7 days I want to delete all spams older than 7 days, > if I simply delete mail files from the file system, is that a 'bad thing' ? > i don't use lmtp so maybe i'm wrong but if you use maildir format just deleting/moving files always work properly From andrzej.filip at gmail.com Tue Jul 10 18:54:12 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Tue, 10 Jul 2012 17:54:12 +0200 Subject: [Dovecot] getting rid of old spam from +spam Maildir ? [ auto purging spambox] In-Reply-To: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> Message-ID: <4FFC5024.8090504@gmail.com> On 07/10/2012 01:25 PM, Voytek Eymont wrote: > I'm trying to setup per user '+spam' delivery from amavis tags, so that > each user gets any mails tagged as spam to 'spam' Maildir via +spam > Dovecot lmtp delivery. > > after say 7 days I want to delete all spams older than 7 days, > if I simply delete mail files from the file system, is that a 'bad thing' ? > [...] IMHO 7 days is dangerously too short. False positives during vacations happen too ;-) I would suggest at least 30 days (as used by Gmail). From sandro.tosi at dada.eu Tue Jul 10 18:54:23 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Tue, 10 Jul 2012 17:54:23 +0200 Subject: [Dovecot] Make a whole mailbox read-only Message-ID: <4FFC502F.2090004@dada.eu> Hello, i've found an old thread [1] where it was stated that a whole mailbox can be made read-only only setting the according permissions on the filesystem side. [1] http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-12/msg00300.html i'd like to check if it's still the only option we have with dovecot v2.0.18 . Regards, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From wojtek at wojtek.tensor.gdynia.pl Tue Jul 10 20:11:09 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 10 Jul 2012 19:11:09 +0200 (CEST) Subject: [Dovecot] getting rid of old spam from +spam Maildir ? [ auto purging spambox] In-Reply-To: <4FFC5024.8090504@gmail.com> References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> <4FFC5024.8090504@gmail.com> Message-ID: >> if I simply delete mail files from the file system, is that a 'bad thing' ? >> [...] > IMHO 7 days is dangerously too short. False positives during vacations > happen too ;-) > > I would suggest at least 30 days (as used by Gmail). > or use spam system that NEVER actually receive something it thinks it is spam, but provide proper SMTP response. From andrzej.filip at gmail.com Tue Jul 10 21:38:44 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Tue, 10 Jul 2012 20:38:44 +0200 Subject: [Dovecot] getting rid of old spam from +spam Maildir ? [ auto purging spambox] In-Reply-To: References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> <4FFC5024.8090504@gmail.com> Message-ID: <4FFC76B4.40804@gmail.com> On 07/10/2012 07:11 PM, Wojciech Puchar wrote: >>> if I simply delete mail files from the file system, is that a 'bad >>> thing' ? >>> [...] >> IMHO 7 days is dangerously too short. False positives during vacations >> happen too ;-) >> >> I would suggest at least 30 days (as used by Gmail). >> > or use spam system that NEVER actually receive something it thinks it > is spam, but provide proper SMTP response. I personally do not recommend rejecting cases (very) close to spam/ham border in SMTP session BUT IMHO accepting to many spams into spambox makes "final check by the recipient" a laughing stock/dangerous illusion. From daniel.parthey at informatik.tu-chemnitz.de Wed Jul 11 01:49:47 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Wed, 11 Jul 2012 00:49:47 +0200 Subject: [Dovecot] doveadm purge -A via doveadm-proxy director fails after some users In-Reply-To: <53B237A0-3A44-47DC-B41A-82CB5D174254@iki.fi> References: <53B237A0-3A44-47DC-B41A-82CB5D174254@iki.fi> Message-ID: <20120710224947.GA10641@daniel.localdomain> Timo Sirainen wrote: > On 29.6.2012, at 19.21, Daniel Parthey wrote: > > > Jun 29 15:40:31 10.129.3.249 dovecot: doveadm(user31 at domain1.example.org): > > Error: user user31 at domain1.example.org: Error reading configuration: > > net_connect_unix(/var/run/dovecot/config) failed: Permission denied > > I've noticed a similar problem happening somewhat randomly, but I still > haven't looked into why exactly it happens. Anyway the attached patch should > fix this specific error, but I'm not sure if there isn't another one. Try and > let me know? :) Unfortunately, the problem still persists with dovecot 2.1.8, which already contains the following code: enum master_service_flags service_flags = MASTER_SERVICE_FLAG_KEEP_CONFIG_OPEN; const char *error; master_service = master_service_init("doveadm", service_flags, &argc, &argv, NULL); if (master_getopt(master_service) > 0) return FATAL_DEFAULT; The command /usr/bin/doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -A still generates the following errors after iterating some dozen users: doveadm(nagios at metaways.de): Error: doveadm server failure doveadm: Error: Failed to iterate through some users Which information should I provide to help debugging the problem? Kind regards Daniel -- https://plus.google.com/103021802792276734820 From ncjeffgus at zimage.com Wed Jul 11 02:47:17 2012 From: ncjeffgus at zimage.com (Jeff Gustafson) Date: Tue, 10 Jul 2012 16:47:17 -0700 Subject: [Dovecot] more dsync problems: Error: proxy client timed out Message-ID: <1341964037.22642.3.camel@maclinux> Hi all, I'm using dsync for backups. I recently upgraded to 2.1.7 due to issues with dsync mirror/backups. 2.1.7 fixed the issue with the guid conflict, but now I'm seeing this error on a couple of mailboxes: Error: proxy client timed out (waiting for output stream to flush, 0 bytes left) If there are 0 bytes left, why is it trying to flush? I'm using this command to backup the mailbox: dsync -vo mail_home=/home/users/user%domain.com backup ssh vmail at 10.1.2.2 dsync -o mail_home=/home/.incoming_mail_migrations/users/user%adomain.com Any ideas? ...Jeff From ncjeffgus at zimage.com Wed Jul 11 02:51:42 2012 From: ncjeffgus at zimage.com (Jeff Gustafson) Date: Tue, 10 Jul 2012 16:51:42 -0700 Subject: [Dovecot] more info (more dsync problems) Message-ID: <1341964302.22642.6.camel@maclinux> Rerunning the dsync command right after the timeout (see previous message) gave me these errors: dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user%domain.com/mdbox/s torage/m.60 (around offset=807789): msg header has bad magic value dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user%domain.com/mdbox/s torage/m.60 (around offset=807789): msg header has bad magic value dsync-local(vmail): Error: remote: dsync-remote(vmail): Warning: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: Inconsistency in map index (2,75904 != 2,82524) dsync-remote(vmail): Warning: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: rebuilding indexes dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage/m.60 (around offset=807789): msg header has bad magic value dsync-local(vmail): Error: remote: dsync-remote(vmail): Warning: dbox: Copy of the broken file saved to /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage/m.60.broken dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage/m.60 (around offset=807789): EOF reading msg header (got 0/30 bytes) dsync-local(vmail): Error: remote: dsync-remote(vmail): Warning: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: rebuilding indexes dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: Duplicate GUID 1341304555.M433317P26661.n13,S=142447,W=144391 in m.62:790375 and m.60:790375 From ewald.lists at fun.de Wed Jul 11 16:42:16 2012 From: ewald.lists at fun.de (Ewald Dieterich) Date: Wed, 11 Jul 2012 15:42:16 +0200 Subject: [Dovecot] mail_log plugin logs UTF-8 mailbox name Message-ID: <4FFD82B8.6030309@fun.de> A small bug in the mail_log plugin: When I copy a mail from the inbox to a mailbox with special characters, the mailbox name is logged in UTF-8: [...] copy from INBOX: box=INBOX.Euro ?, [...] I guess the name should be in mUTF-7, like it is done when I copy the other way round: [...] copy from Euro &IKw-: box=INBOX, [...] From tss at iki.fi Wed Jul 11 20:11:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 11 Jul 2012 19:11:26 +0200 Subject: [Dovecot] mail_log plugin logs UTF-8 mailbox name In-Reply-To: <4FFD82B8.6030309@fun.de> References: <4FFD82B8.6030309@fun.de> Message-ID: <2A5F8DC1-EFA2-4956-B103-4D39985C6E69@iki.fi> On 11.7.2012, at 15.42, Ewald Dieterich wrote: > A small bug in the mail_log plugin: > > When I copy a mail from the inbox to a mailbox with special characters, > the mailbox name is logged in UTF-8: > > [...] copy from INBOX: box=INBOX.Euro ?, [...] I think the UTF8 name is more correct here > I guess the name should be in mUTF-7, like it is done when I copy the > other way round: > > [...] copy from Euro &IKw-: box=INBOX, [...] and I should change this instead. Just because IMAP protocol uses mUTF7 doesn't mean that it should be visible anywhere else. (You can already use UTF8 for mailbox names on the filesystem.) From list at airstreamcomm.net Wed Jul 11 21:10:40 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 11 Jul 2012 13:10:40 -0500 Subject: [Dovecot] MySQL dict issues Message-ID: <4FFDC1A0.6040603@airstreamcomm.net> Testing using the mysql dict for quota storage (dovecot-2.0.12-2_127.el5), and ran into a couple issues. First is a permissions issue: dovecot: lmtp(26786, dude at test.tld): Error: net_connect_unix(/var/run/dovecot/dict) failed: Permission denied (euid=5000() egid=5000() missing +r perm: /var/run/dovecot/dict, euid is not dir owner) The file /var/run/dovecot/dict is listed with srw------- perms. I changes the perms to srw----rwx and it works, but I would like to figure out how to configure properly to run with the uid and gid 5000 as this is the user for virtual mail. The second issue is that two identical rows are created in the mysql database for each user quota. The next message delivered updates both rows. Here is my quota config: 90-quota.conf: plugin { quota_rule = *:storage=10G quota_rule2 = Trash:storage=+10% } plugin { quota = dict:user::proxy::userquota } dict { userquota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } dovecot-dict-sql.conf.ext: connect = host=a.b.c.d dbname=mail user=user password=derp map { pattern = priv/quota/storage table = domain_user_quotas username_field = username value_field = bytes } map { pattern = priv/quota/messages table = domain_user_quotas username_field = username value_field = message_count } The last issue might be a documentation omission. Dovecot warned me that the mysql user did not have DELETE privileges on the quota table: dovecot: dict: Error: sql dict: commit failed: DELETE command denied to user 'user'@'a.b.c.d' for table 'domain_user_quotas' The quota dict documentation (at http://wiki2.dovecot.org/Quota/Dict) does not mention DELETE being needed: MySQL uses the following queries to update the quota. You need suitable privileges. INSERT INTO table (bytes,username) VALUES ('112497180','foo at spam.dom') ON DUPLICATE KEY UPDATE bytes='112497180'; INSERT INTO table (messages,username) VALUES ('1743','foo at spam.dom') ON DUPLICATE KEY UPDATE messages='1743'; UPDATE table SET bytes=bytes-14433,messages=messages-2 WHERE username = 'foo at spam.dom'; From ewald.lists at fun.de Wed Jul 11 21:41:02 2012 From: ewald.lists at fun.de (Ewald Dieterich) Date: Wed, 11 Jul 2012 20:41:02 +0200 Subject: [Dovecot] mail_log plugin logs UTF-8 mailbox name In-Reply-To: <2A5F8DC1-EFA2-4956-B103-4D39985C6E69@iki.fi> References: <4FFD82B8.6030309@fun.de> <2A5F8DC1-EFA2-4956-B103-4D39985C6E69@iki.fi> Message-ID: <4FFDC8BE.9060600@fun.de> On 07/11/12 19:11, Timo Sirainen wrote: > On 11.7.2012, at 15.42, Ewald Dieterich wrote: > >> A small bug in the mail_log plugin: >> >> [...] copy from INBOX: box=INBOX.Euro ?, [...] > > I think the UTF8 name is more correct here > >> [...] copy from Euro &IKw-: box=INBOX, [...] > > and I should change this instead. Here is another one: $ doveadm mailbox create -u 520000002149-0001 "INBOX.Umlaut ?" doveadm(520000002149-0001): Info: Mailbox created: Umlaut &AOQ- And again from the mail_log plugin: [...] Mailbox deleted: Umlaut &AOQ- > Just because IMAP protocol uses mUTF7 doesn't mean that it should be > visible anywhere else. Well, it used to be mUTF-7 everywhere, and now all the scripts that use doveadm are broken. Also, it used to be so easy to go from a protocol snippet to the filesystem and then to a doveadm command line. Just copy and paste. But I'm just complaining ;-) > (You can already use UTF8 for mailbox names on the filesystem.) You mean there is a configuration option in Dovecot? I didn't know that. From sandro.tosi at dada.eu Thu Jul 12 11:05:53 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Thu, 12 Jul 2012 10:05:53 +0200 Subject: [Dovecot] Make a whole mailbox read-only In-Reply-To: <4FFC502F.2090004@dada.eu> References: <4FFC502F.2090004@dada.eu> Message-ID: <4FFE8561.2070608@dada.eu> Could someone please give it a look? Timo? :) Cheers, Sandro On 07/10/2012 05:54 PM, Sandro Tosi wrote: > Hello, > i've found an old thread [1] where it was stated that a whole mailbox > can be made read-only only setting the according permissions on the > filesystem side. > > [1] > http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-12/msg00300.html > > > i'd like to check if it's still the only option we have with dovecot > v2.0.18 . > > Regards, -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From Nick at 5sg.eu Thu Jul 12 09:12:51 2012 From: Nick at 5sg.eu (Nick at 5sg.eu) Date: Thu, 12 Jul 2012 02:12:51 -0400 Subject: [Dovecot] Maildir messages Message-ID: <4FFE6AE3.2070003@5sg.eu> Hi all, Hopefully, I'm sending this to the right place... Can someone guide me on changing the Maildir messages files back to the single file format I used to have instead of the dovecot.index /cur /tmp /new format my newer server currently has. I'm running dovecot v1.2.9 postfix v2.7 on Ubuntu v10.4 Thanks, Nick. . . . . MU/Webcasting -- _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ . . . . . . . . Please join my Tech MeetUp group that meets in the NYC metro area Webcasting and Video Streaming Network: http://TheWebcasting.net . . . . . . . . _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ From robert at schetterer.org Thu Jul 12 12:00:31 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 12 Jul 2012 11:00:31 +0200 Subject: [Dovecot] Maildir messages In-Reply-To: <4FFE6AE3.2070003@5sg.eu> References: <4FFE6AE3.2070003@5sg.eu> Message-ID: <4FFE922F.7000700@schetterer.org> Am 12.07.2012 08:12, schrieb Nick at 5sg.eu: > Hi all, > Hopefully, I'm sending this to the right place... > > Can someone guide me on changing the Maildir messages files > back to the single file format I used to have instead of the > dovecot.index /cur /tmp /new format my newer server currently > has. I'm running dovecot v1.2.9 postfix v2.7 on Ubuntu v10.4 > > Thanks, > Nick. . . . . > MU/Webcasting > -- > _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ > . . . . . . . . > Please join my Tech MeetUp group that meets in the NYC metro area > Webcasting and Video Streaming Network: http://TheWebcasting.net > . . . . . . . . > _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ perhaps look here http://wiki2.dovecot.org/MailLocation -- Best Regards MfG Robert Schetterer From robert at schetterer.org Thu Jul 12 12:04:33 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 12 Jul 2012 11:04:33 +0200 Subject: [Dovecot] Make a whole mailbox read-only In-Reply-To: <4FFE8561.2070608@dada.eu> References: <4FFC502F.2090004@dada.eu> <4FFE8561.2070608@dada.eu> Message-ID: <4FFE9321.1010006@schetterer.org> Am 12.07.2012 10:05, schrieb Sandro Tosi: > Could someone please give it a look? Timo? :) perhaps try ideas from http://wiki2.dovecot.org/HowTo/ReadOnlyArchive?highlight=%28read%29|%28only%29 > > Cheers, > Sandro > > On 07/10/2012 05:54 PM, Sandro Tosi wrote: >> Hello, >> i've found an old thread [1] where it was stated that a whole mailbox >> can be made read-only only setting the according permissions on the >> filesystem side. >> >> [1] >> http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-12/msg00300.html >> >> >> >> i'd like to check if it's still the only option we have with dovecot >> v2.0.18 . >> >> Regards, > -- Best Regards MfG Robert Schetterer From wojtek at wojtek.tensor.gdynia.pl Thu Jul 12 13:29:18 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 12 Jul 2012 12:29:18 +0200 (CEST) Subject: [Dovecot] getting rid of old spam from +spam Maildir ? [ auto purging spambox] In-Reply-To: <4FFC76B4.40804@gmail.com> References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> <4FFC5024.8090504@gmail.com> <4FFC76B4.40804@gmail.com> Message-ID: >> is spam, but provide proper SMTP response. > > I personally do not recommend rejecting cases (very) close to spam/ham > border in SMTP session well i have very very low amount of misses but consider generating meaningful SMTP response like "if you are not spammer call us, so we will fix it" From nick.z.edwards at gmail.com Thu Jul 12 13:32:01 2012 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Thu, 12 Jul 2012 20:32:01 +1000 Subject: [Dovecot] bcypt availability Message-ID: Dear Timo, Do you intend to introduce bcrypt into the built in password schemes? In lew of all these hacks lately many larger companies appear moving this way, we are looking at it too, but dovecot will then be the weakest link in the database security. So, are you planning on this and if so what sort of timeframe / version would you expect it to be in beta ? Nik From wojtek at wojtek.tensor.gdynia.pl Thu Jul 12 13:32:45 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 12 Jul 2012 12:32:45 +0200 (CEST) Subject: [Dovecot] Maildir messages In-Reply-To: <4FFE6AE3.2070003@5sg.eu> References: <4FFE6AE3.2070003@5sg.eu> Message-ID: > Hopefully, I'm sending this to the right place... > > Can someone guide me on changing the Maildir messages files > back to the single file format I used to have instead of the > dovecot.index /cur /tmp /new format my newer server currently > has. I'm running dovecot v1.2.9 postfix v2.7 on Ubuntu v10.4 i don't really understand why you want to do it, single file mailboxes are plain inefficient. maybe - just as in article below http://www.bishnet.net/tim/blog/tag/maildir-to-mbox/ you like archive in such format From andras at lay.hu Thu Jul 12 14:13:21 2012 From: andras at lay.hu (Lay =?ISO-8859-1?B?QW5kcuFz?=) Date: Thu, 12 Jul 2012 13:13:21 +0200 Subject: [Dovecot] Seq-range order in sequence-set are matter? Message-ID: <40f28e5c42b.4117456b@smtp.gmail.com> Hi! I have a mailbox with 12 messages. Theese two commands: x fetch 5,8:12 (UID) * 5 FETCH (UID 10) * 8 FETCH (UID 13) * 9 FETCH (UID 14) * 10 FETCH (UID 15) * 11 FETCH (UID 16) * 12 FETCH (UID 17) x OK Fetch completed. and x fetch 8:12,5 (UID) * 5 FETCH (UID 10) * 8 FETCH (UID 13) * 9 FETCH (UID 14) * 10 FETCH (UID 15) * 11 FETCH (UID 16) * 12 FETCH (UID 17) x OK Fetch completed. outputs are equeal, this clean. But: x fetch 1:*,5 (UID) * 1 FETCH (UID 4) * 2 FETCH (UID 5) * 3 FETCH (UID 6) * 4 FETCH (UID 7) * 5 FETCH (UID 10) * 6 FETCH (UID 11) * 7 FETCH (UID 12) * 8 FETCH (UID 13) * 9 FETCH (UID 14) * 10 FETCH (UID 15) * 11 FETCH (UID 16) * 12 FETCH (UID 17) x OK Fetch completed. and x fetch 5,1:* (UID) * 5 FETCH (UID 10) * 6 FETCH (UID 11) * 7 FETCH (UID 12) * 8 FETCH (UID 13) * 9 FETCH (UID 14) * 10 FETCH (UID 15) * 11 FETCH (UID 16) * 12 FETCH (UID 17) x OK Fetch completed. command outputs are NOT equal? Why? Thank you! Lay From amateo at um.es Thu Jul 12 14:14:37 2012 From: amateo at um.es (Angel L. Mateo) Date: Thu, 12 Jul 2012 13:14:37 +0200 Subject: [Dovecot] doveadm director move behaviour Message-ID: <4FFEB19D.6020105@um.es> Hi, I have a director farm running in front of dovecot pop/imap servers. For a reason, one of my backend servers have been down, so poolmon disables it in director servers and users corresponding to it has been sent to other servers. Until this, no problem. Now my failed backend server is up again, and I want to redirect all of its users to it, without waiting for it director sessions to end. So I have used doveadm director move to move them to its "hashed" server. Although a doveadm director status now shows me that this user is directed to the new server, doveadm who in the temporal server shows me that the user is actually connected to it, although messages are delivered to it through is correct server. So I have had to manually kick that users in the temporal servers. Is this the expected behaviour of the move command for director? Is there any way to also force the kick of the imap/pop user's connection in the temporal server? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337 From andrzej.filip at gmail.com Thu Jul 12 15:58:24 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Thu, 12 Jul 2012 14:58:24 +0200 Subject: [Dovecot] getting rid of old spam from +spam Maildir ? [ auto purging spambox] In-Reply-To: References: <3ee8ea3983f6abae778115c9c1eda367.squirrel@geko.sbt.net.au> <4FFC5024.8090504@gmail.com> <4FFC76B4.40804@gmail.com> Message-ID: <4FFEC9F0.4050205@gmail.com> On 07/12/2012 12:29 PM, Wojciech Puchar wrote: >>> is spam, but provide proper SMTP response. >> >> I personally do not recommend rejecting cases (very) close to spam/ham >> border in SMTP session > well i have very very low amount of misses > > but consider generating meaningful SMTP response like "if you are not > spammer call us, so we will fix it" 1) I personally do not like using ONLY binary choice [accept/reject (in SMTP session)]. I personally do prefer something for (small) range in between. 2) IMHO senders' opinions are much less relevant than recipients' opinions for postmaster of receiving server. [received e.g. via antispam plug-in of dovecot (moving messages in/out spambox folder] From veiko.kukk at ekp.ee Thu Jul 12 16:43:04 2012 From: veiko.kukk at ekp.ee (Veiko Kukk) Date: Thu, 12 Jul 2012 16:43:04 +0300 Subject: [Dovecot] Override userdb=passwd gid Message-ID: <4FFED468.5000301@ekp.ee> Hi! I have problem with Dovecot 2.1.1 writing maildir files as users primary group. Tried to override gid value, but probably i'm missing something... doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-308.11.1.el5 x86_64 CentOS release 5.8 (Final) auth_verbose = yes info_log_path = /var/log/dovecot/info.log listen = * log_path = /var/log/dovecot/error.log mail_location = maildir:%h/Maildir passdb { args = cache_key=%u dovecot driver = pam } protocols = imap service imap-login { process_min_avail = 40 service_count = 1 } service imap { process_limit = 2048 vsz_limit = 2 G } ssl = required ssl_cert = dovecot-2.1.8 sieve-0.3.1 binutils-2.22 When dovecot and sieve is linked with ld.gold I get a segfault in checkscript command: $ telnet localhost 4190 [some output] authenticate "PLAIN" "" [some more output] checkscript "foobarscript" connection closed Logs: 2012-07-12T19:30:23.624158+00:00 bob dovecot: master: Dovecot v2.1.8 starting up 2012-07-12T19:31:22.548913+00:00 bob dovecot: managesieve-login: Login: user=, method=PLAIN, rip=192.168.122.1, lip=192.168.122.64, mpid=12169, session= 2012-07-12T19:31:34.636747+00:00 bob dovecot: managesieve(username): Fatal: master: service(managesieve): child 12169 killed with signal 11 (core dumped) make check also fails (thank you for that): [...] Test case: ./tests/execute/errors.svtest: 1: Test 'Action conflicts: reject <-> fileinto' SUCCEEDED 2: Test 'Action conflicts: reject <-> keep' SUCCEEDED 3: Test 'Action conflicts: reject <-> redirect' SUCCEEDED 4: Test 'Action limit' SUCCEEDED 5: Test 'Redirect limit' SUCCEEDED /tmp/work/dovecot-2.1-pigeonhole-0.3.1/src/testsuite/.libs/lt-testsuite: symbol lookup error: /tmp/work/dovecot-2.1-pigeonhole-0.3.1/src/lib-sieve/.libs/libdovecot-sieve.so.0: undefined symbol: mail_deliver_get_log_var_expand_table make: *** [tests/execute/errors.svtest] Error 127 Backtrace is attached. -- Eray Aslan -------------- next part -------------- [New LWP 12169] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `dovecot/managesieve'. Program terminated with signal 11, Segmentation fault. #0 0x00007f9e154a2aa6 in ?? () from /lib64/libc.so.6 #0 0x00007f9e154a2aa6 in ?? () from /lib64/libc.so.6 No symbol table info available. #1 0x000000000040b353 in sieve_storage_save_will_activate (ctx=0x17ee170) at sieve-storage-save.c:329 _data_stack_cur_id = 5 scriptname = 0x17b8356 "foobarscript.sieve" ret = #2 0x0000000000406170 in cmd_putscript_finish_parsing (cmd=0x17e4c88) at cmd-putscript.c:206 ehandler = cpflags = 3 sbin = errors = _data_stack_cur_id = 4 script = 0x17ef180 success = true client = 0x17e4bf0 ctx = 0x17e7150 args = 0x17c3c28 ret = #3 0x0000000000406407 in cmd_putscript_continue_script (cmd=0x17e4c88) at cmd-putscript.c:423 failed = all_written = client = 0x17e4bf0 ctx = 0x17e7150 size = 18446744073709551615 ret = #4 0x0000000000406837 in cmd_putscript_continue_parsing (cmd=0x17e4c88) at cmd-putscript.c:332 client = 0x17e4bf0 ctx = 0x17e7150 args = 0x17c3c28 ret = #5 0x0000000000408597 in client_handle_input (cmd=0x17e4c88) at managesieve-client.c:546 client = 0x17e4bf0 __FUNCTION__ = "client_handle_input" #6 0x000000000040863f in client_handle_input (cmd=0x17e4c88) at managesieve-client.c:598 client = 0x17e4bf0 __FUNCTION__ = "client_handle_input" #7 0x00000000004087da in client_input (client=0x17e4bf0) at managesieve-client.c:639 _data_stack_cur_id = 3 cmd = 0x17e4c88 ret = #8 0x00007f9e157cfd16 in io_loop_call_io (io=0x17e4fc0) at ioloop.c:379 ioloop = 0x17c0680 t_id = 2 #9 0x00007f9e157d0a37 in io_loop_handler_run (ioloop=) at ioloop-epoll.c:213 ctx = 0x17c09f0 events = 0x0 event = 0x17c0a60 list = 0x17dba30 io = tv = {tv_sec = 1799, tv_usec = 999667} msecs = ret = 1 i = call = #10 0x00007f9e157cf728 in io_loop_run (ioloop=0x17c0680) at ioloop.c:398 No locals. #11 0x00007f9e157bbc43 in master_service_run (service=0x17c0530, callback=) at master-service.c:543 No locals. #12 0x00000000004094f5 in main (argc=1, argv=0x17c0370) at main.c:315 set_roots = {0x412b20, 0x0} login_set = {auth_socket_path = 0x17b8088 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x408fc0 , failure_callback = 0x408f70 } service_flags = storage_service_flags = username = c = From voytek at sbt.net.au Fri Jul 13 02:03:07 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Fri, 13 Jul 2012 09:03:07 +1000 Subject: [Dovecot] ot: execute a script via email? Message-ID: anyone has any tips what's best way to execute a script via email ? I have a wget script that fetches some pages, and, emails me the output, there is no paramters passed, it's all in the script I'll like to email in and action the script to get the results email -- Voytek From h.reindl at thelounge.net Fri Jul 13 02:21:24 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 13 Jul 2012 01:21:24 +0200 Subject: [Dovecot] ot: execute a script via email? In-Reply-To: References: Message-ID: <4FFF5BF4.8060708@thelounge.net> Am 13.07.2012 01:03, schrieb Voytek Eymont: > anyone has any tips what's best way to execute a script via email ? > > I have a wget script that fetches some pages, and, emails me the output, > there is no paramters passed, it's all in the script > I'll like to email in and action the script to get the results email postfix and transports may do this IMHO not a job of dovecot as LDA/mailstorage -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From stan at hardwarefreak.com Fri Jul 13 03:49:46 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 12 Jul 2012 19:49:46 -0500 Subject: [Dovecot] Maildir messages In-Reply-To: References: <4FFE6AE3.2070003@5sg.eu> Message-ID: <4FFF70AA.3080103@hardwarefreak.com> On 7/12/2012 5:32 AM, Wojciech Puchar wrote: > i don't really understand why you want to do it, single file mailboxes > are plain inefficient. mbox is more efficient than maildir for many common operations, specifically from a filesystem IO perspective: 1. new mail delivery 2. searching a mailbox folder 3. mass deletion of emails from one mailbox folder maildir is more IO efficient when reading and deleting individual emails. mbox puts the load on the mail server application and on memory. maildir puts the load on the IO subsysetem. Which is precisely why Timo created the mdbox mail storage format, attempting to get the best of both worlds. -- Stan From ka at pacific.net Fri Jul 13 04:42:18 2012 From: ka at pacific.net (Ken Anderson) Date: Thu, 12 Jul 2012 20:42:18 -0500 Subject: [Dovecot] ot: execute a script via email? In-Reply-To: References: Message-ID: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> Old school - in /etc/aliases you can set: user "|/home/user/script" And read the mail from stdin. Ken Voytek Eymont wrote: >anyone has any tips what's best way to execute a script via email ? > >I have a wget script that fetches some pages, and, emails me the >output, >there is no paramters passed, it's all in the script >I'll like to email in and action the script to get the results email > > >-- >Voytek Pacific.net From dmalolepszy at optusnet.com.au Fri Jul 13 08:30:10 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Fri, 13 Jul 2012 15:30:10 +1000 Subject: [Dovecot] SQLite dovecot query caching In-Reply-To: <1340477828.5967.91.camel@hurina> References: <4FE5FEB4.2050801@optusnet.com.au> <1340473147.5967.89.camel@hurina> <4FE608E8.6090106@optusnet.com.au> <1340477828.5967.91.camel@hurina> Message-ID: <4FFFB262.7040409@optusnet.com.au> On 24/06/12 04:57, Timo Sirainen wrote: > On Sun, 2012-06-24 at 04:20 +1000, Dominic Malolepszy wrote: >> On 24/06/12 3:39 AM, Timo Sirainen wrote: >>> On Sun, 2012-06-24 at 03:36 +1000, Dominic Malolepszy wrote: >>>> Hi, >>>> >>>> I am wondering if Dovecot caches SQLite queries, and how well it works >>>> in high performance setups. I am particularly interested because in the >>>> below thread SQLite has been suggested as a means of Dovecot proxying >>>> connections to different ports. >>> You can enable auth cache: >>> http://wiki2.dovecot.org/Authentication/Caching >>> >>> >> >> This is a per user caching though, it will still have to perform a sql >> look up each time a unique user authenticates to determine what port the >> proxy should forward each connection. Is that accurate? > > It caches the passdb lookup. The cache key consists of the given % > variables in the SQL query. So if your SQL query doesn't contain %n/%u > then the cache doesn't add per-user entries. > > Hi, I had a chance to play around with this in the lab. The cache hit appears to be returning values that should actually be ignored particularly the username for the previous cached entry. This is causing it to rewrite the username using what is in the cache. The passdb config is as follows: passdb { driver = sql args = /etc/dovecot-director/dovecot-sql.conf } The dovecot-sql.conf config contains: driver = sqlite connect = /tmp/dovecot-sqlite.db password_query = select 'y' as proxy, \ NULL AS password, 'y' as nopassword, \ NULL AS user, NULL AS username, \ case '%a' \ when '110' then '9110' \ when '995' then '9995' \ when '143' then '9143' \ when '993' then '9993' end \ as port; To demonstrate I log into POP3 first as user 'dmmailtest250' which populates the cache with (#011proxy#011port=9110#011user=dmmailtest250 at mailtest.com.au#011nopassword=y). Than I log into POP3 as non existent user 'blahtest01229', which causes the passdb to return user=dmmailtest250. Below is what is captured in the logs: Jul 13 14:16:59 mail01 dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011secured#011lip=127.0.0.1#011rip=127.0.0.1#011lport=110#011rport=52873#011resp=AGJsYWh0ZXN0MDEyMjkAcGFzc3dvcmQ= Jul 13 14:16:59 mail01 dovecot: auth: Debug: cache(blahtest01229 at mailtest.com.au,127.0.0.1): hit: #011proxy#011port=9110#011user=dmmailtest250 at mailtest.com.au#011nopassword=y Jul 13 14:16:59 mail01 dovecot: auth: cache(blahtest01229 at mailtest.com.au,127.0.0.1): NULL password access Jul 13 14:16:59 mail01 dovecot: auth: Debug: auth(blahtest01229 at mailtest.com.au,127.0.0.1): username changed blahtest01229 at mailtest.com.au -> dmmailtest250 at mailtest.com.au Jul 13 14:16:59 mail01 dovecot: auth: Debug: client out: OK#0111#011user=dmmailtest250 at mailtest.com.au#011proxy#011port=9110#011pass=password Jul 13 14:16:59 mail01 dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=pop3#011secured#011lip=192.168.1.151#011rip=192.168.1.151#011lport=9110#011rport=60096#011resp=AGRtb2llbWFpbHRlc3QyNTBAb3B0dXNuZXQuY29tLmF1AHBhc3N3b3Jk Jul 13 14:16:59 mail01 dovecot: auth: Debug: cache(dmmailtest250 at mailtest.com.au,192.168.1.151): hit: {crypt}$1$eKFvOhwW$PYfPhY05SNUIRD439USMH/#011user=dmmailtest250 at mailtest.com.au#011userdb_home=/m5/dm/dmmailtest250#011userdb_quota_rule=*:backend=52428800S I have not specified any sort of cache_key explicitly, because according to http://wiki2.dovecot.org/Authentication/Caching "For SQL and LDAP lookups Dovecot figures this out automatically by using all the used %variables as the cache key", which means that %a is the key ie the lport number which is 110 in this case. I tried specifying a NULL user and username, but that does not help. Cheers, Dominic. From sandro.tosi at dada.eu Fri Jul 13 09:46:49 2012 From: sandro.tosi at dada.eu (Sandro Tosi) Date: Fri, 13 Jul 2012 08:46:49 +0200 Subject: [Dovecot] Make a whole mailbox read-only In-Reply-To: <4FFE9321.1010006@schetterer.org> References: <4FFC502F.2090004@dada.eu> <4FFE8561.2070608@dada.eu> <4FFE9321.1010006@schetterer.org> Message-ID: <4FFFC459.7030807@dada.eu> Thanks for the hint for the first 2 points of the howto are false on our system: * we use docevot 2 * we have mailbox in maildir format Regards, Sandro On 07/12/2012 11:04 AM, Robert Schetterer wrote: > Am 12.07.2012 10:05, schrieb Sandro Tosi: >> Could someone please give it a look? Timo? :) > > perhaps try ideas from > > http://wiki2.dovecot.org/HowTo/ReadOnlyArchive?highlight=%28read%29|%28only%29 >> >> Cheers, >> Sandro >> >> On 07/10/2012 05:54 PM, Sandro Tosi wrote: >>> Hello, >>> i've found an old thread [1] where it was stated that a whole mailbox >>> can be made read-only only setting the according permissions on the >>> filesystem side. >>> >>> [1] >>> http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-12/msg00300.html >>> >>> >>> >>> i'd like to check if it's still the only option we have with dovecot >>> v2.0.18 . >>> >>> Regards, >> > > -- Sandro Tosi Product Engineer Shared Hosting Products R&D | Dada.pro eml sandro.tosi at register.it From robert at schetterer.org Fri Jul 13 10:16:55 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Jul 2012 09:16:55 +0200 Subject: [Dovecot] Make a whole mailbox read-only In-Reply-To: <4FFFC459.7030807@dada.eu> References: <4FFC502F.2090004@dada.eu> <4FFE8561.2070608@dada.eu> <4FFE9321.1010006@schetterer.org> <4FFFC459.7030807@dada.eu> Message-ID: <4FFFCB67.9050903@schetterer.org> Am 13.07.2012 08:46, schrieb Sandro Tosi: > Thanks for the hint for the first 2 points of the howto are false on our > system: > > * we use docevot 2 > * we have mailbox in maildir format i wrote about ,take ideas, from there, for sure you have to fit stuff to your needs ,ever > > Regards, > Sandro > > On 07/12/2012 11:04 AM, Robert Schetterer wrote: >> Am 12.07.2012 10:05, schrieb Sandro Tosi: >>> Could someone please give it a look? Timo? :) >> >> perhaps try ideas from >> >> http://wiki2.dovecot.org/HowTo/ReadOnlyArchive?highlight=%28read%29|%28only%29 >> >>> >>> Cheers, >>> Sandro >>> >>> On 07/10/2012 05:54 PM, Sandro Tosi wrote: >>>> Hello, >>>> i've found an old thread [1] where it was stated that a whole mailbox >>>> can be made read-only only setting the according permissions on the >>>> filesystem side. >>>> >>>> [1] >>>> http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-12/msg00300.html >>>> >>>> >>>> >>>> >>>> i'd like to check if it's still the only option we have with dovecot >>>> v2.0.18 . >>>> >>>> Regards, >>> >> >> > -- Best Regards MfG Robert Schetterer From andrzej.filip at gmail.com Fri Jul 13 11:43:31 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Fri, 13 Jul 2012 10:43:31 +0200 Subject: [Dovecot] ot: execute a script via email? In-Reply-To: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> References: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> Message-ID: <4FFFDFB3.3060700@gmail.com> On 07/13/2012 03:42 AM, Ken Anderson wrote: > > Voytek Eymont wrote: > >> anyone has any tips what's best way to execute a script via email ? >> >> I have a wget script that fetches some pages, and, emails me the >> output, >> there is no paramters passed, it's all in the script >> I'll like to email in and action the script to get the results email >> >> >> -- >> Voytek > Old school - in /etc/aliases you can set: > user "|/home/user/script" > And read the mail from stdin. > Ken There is available dovecot specific sieve extension (vnd.dovecot.pipe ) to pipe messages to a pre-defined set of external programs. http://wiki2.dovecot.org/Pigeonhole/Sieve From amateo at um.es Fri Jul 13 11:52:42 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 13 Jul 2012 10:52:42 +0200 Subject: [Dovecot] Difference between proxy_refresh and proxy_timeout in director Message-ID: <4FFFE1DA.7020706@um.es> Hello, I know proxy_timeout is the timeout for the director operation. But what is the purpose for proxy_refresh? -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n _o) y las Comunicaciones Aplicadas (ATICA) / \\ http://www.um.es/atica _(___V Tfo: 868887590 Fax: 868888337 From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 12:09:56 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 11:09:56 +0200 (CEST) Subject: [Dovecot] Maildir messages In-Reply-To: <4FFF70AA.3080103@hardwarefreak.com> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> Message-ID: > specifically from a filesystem IO perspective: > > 1. new mail delivery not much difference. > 2. searching a mailbox folder if you do linear search then yes > 3. mass deletion of emails from one mailbox folder > 3 is not true. > maildir is more IO efficient when reading and deleting individual emails. > and making backups. deleting from maildir means just delete a file, not shuffle data. Everyone needs to make backups while it is unlinkely that anyone like every backup to be effectively full backup. > mbox puts the load on the mail server application and on memory. and on I/O too - often quite a bit > maildir puts the load on the IO subsysetem. Which is precisely why Timo > created the mdbox mail storage format, attempting to get the best of > both worlds. And this is great idea and actually works :) mbox may make sense for archive storage. you create archive folder once and never modify anything From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 12:14:19 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 11:14:19 +0200 (CEST) Subject: [Dovecot] ot: execute a script via email? In-Reply-To: References: Message-ID: > anyone has any tips what's best way to execute a script via email ? use procmail - it's easy and actually works. you can do basically everything with procmail. This question is off topic, dovecot is not SMTP agent. > I have a wget script that fetches some pages, and, emails me the output, > there is no paramters passed, it's all in the script > I'll like to email in and action the script to get the results email > use procmail and possibly formail to execute anything and optionally reply the result man procmailrc man procmailex man formail make sure procmail is actually executed on your mail system. depends of your MTA. From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 12:14:56 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 11:14:56 +0200 (CEST) Subject: [Dovecot] ot: execute a script via email? In-Reply-To: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> References: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> Message-ID: > Old school - in /etc/aliases you can set: > user "|/home/user/script" > And read the mail from stdin. > Ken works great but requires root to do this > > > Voytek Eymont wrote: > >> anyone has any tips what's best way to execute a script via email ? >> >> I have a wget script that fetches some pages, and, emails me the >> output, >> there is no paramters passed, it's all in the script >> I'll like to email in and action the script to get the results email >> >> >> -- >> Voytek > > Pacific.net > From amateo at um.es Fri Jul 13 12:37:08 2012 From: amateo at um.es (Angel L. Mateo) Date: Fri, 13 Jul 2012 11:37:08 +0200 Subject: [Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument In-Reply-To: <4FFAD095.4050804@schetterer.org> References: <4FFAC2F6.4040001@um.es> <1341834686.19913.27.camel@hurina> <4FFAD095.4050804@schetterer.org> Message-ID: <4FFFEC44.1050002@um.es> El 09/07/12 14:37, Robert Schetterer escribi?: > guess this is latest for 12.04 > http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/ > I have check this 3.4 kernel and it seems there is no problem with it -- Angel L. Mateo Mart?nez Secci?n de Telem?tica ?rea de Tecnolog?as de la Informaci?n _o) y las Comunicaciones Aplicadas (ATICA) / \\ http://www.um.es/atica _(___V Tfo: 868887590 Fax: 868888337 From robert at schetterer.org Fri Jul 13 12:44:22 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 13 Jul 2012 11:44:22 +0200 Subject: [Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument In-Reply-To: <4FFFEC44.1050002@um.es> References: <4FFAC2F6.4040001@um.es> <1341834686.19913.27.camel@hurina> <4FFAD095.4050804@schetterer.org> <4FFFEC44.1050002@um.es> Message-ID: <4FFFEDF6.2090508@schetterer.org> Am 13.07.2012 11:37, schrieb Angel L. Mateo: > El 09/07/12 14:37, Robert Schetterer escribi?: > >> guess this is latest for 12.04 >> http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4-precise/ >> > I have check this 3.4 kernel and it seems there is no problem with it > cool, thx for info -- Best Regards MfG Robert Schetterer From stephan at rename-it.nl Fri Jul 13 12:53:30 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Fri, 13 Jul 2012 11:53:30 +0200 Subject: [Dovecot] managesieve segfault with gold linker In-Reply-To: <4FFF3326.9060802@caf.com.tr> References: <4FFF3326.9060802@caf.com.tr> Message-ID: <4FFFF01A.9020402@rename-it.nl> On 7/12/2012 10:27 PM, Eray Aslan wrote: > dovecot-2.1.8 sieve-0.3.1 binutils-2.22 > > When dovecot and sieve is linked with ld.gold I get a segfault in > checkscript command: > > $ telnet localhost 4190 > [some output] > authenticate "PLAIN" "" > [some more output] > checkscript "foobarscript" > connection closed > > Logs: > 2012-07-12T19:30:23.624158+00:00 bob dovecot: master: Dovecot v2.1.8 > starting up > 2012-07-12T19:31:22.548913+00:00 bob dovecot: managesieve-login: Login: > user=, method=PLAIN, rip=192.168.122.1, lip=192.168.122.64, > mpid=12169, session= > 2012-07-12T19:31:34.636747+00:00 bob dovecot: managesieve(username): > Fatal: master: service(managesieve): child 12169 killed with signal 11 > (core dumped) Based on your excellent bug report, I fixed this one yesterday: http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/6ceeb6421231 > make check also fails (thank you for that): > [...] > Test case: ./tests/execute/errors.svtest: > > 1: Test 'Action conflicts: reject <-> fileinto' SUCCEEDED > 2: Test 'Action conflicts: reject <-> keep' SUCCEEDED > 3: Test 'Action conflicts: reject <-> redirect' SUCCEEDED > 4: Test 'Action limit' SUCCEEDED > 5: Test 'Redirect limit' SUCCEEDED > /tmp/work/dovecot-2.1-pigeonhole-0.3.1/src/testsuite/.libs/lt-testsuite: > symbol lookup error: > /tmp/work/dovecot-2.1-pigeonhole-0.3.1/src/lib-sieve/.libs/libdovecot-sieve.so.0: > undefined symbol: mail_deliver_get_log_var_expand_table > make: *** [tests/execute/errors.svtest] Error 127 I can't reproduce this (on Debian stable). How exactly did you configure dovecot sources to use ld.gold? I just installed the binutils-gold package, but I get the impression that ld.gold may not actually be used. Also, perhaps this is caused by a version difference (mine is GNU gold (GNU Binutils for Debian 2.20.1-system.20100303) 1.9). I get the impression that this is an LD bug, but I do like to be sure. Regards, Stephan. From ka at pacific.net Fri Jul 13 14:28:14 2012 From: ka at pacific.net (Ken Anderson) Date: Fri, 13 Jul 2012 06:28:14 -0500 Subject: [Dovecot] ot: execute a script via email? In-Reply-To: References: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> Message-ID: If you don't have root, you are probably going to be restricted from other methods by the security policy of a shared host. In that case, just use fetchmail or other means of checking mail for your 'trigger' message, then run your process where you DO have permission. Ken Wojciech Puchar wrote: >> Old school - in /etc/aliases you can set: >> user "|/home/user/script" >> And read the mail from stdin. >> Ken > >works great but requires root to do this > >> >> >> Voytek Eymont wrote: >> >>> anyone has any tips what's best way to execute a script via email ? >>> >>> I have a wget script that fetches some pages, and, emails me the >>> output, >>> there is no paramters passed, it's all in the script >>> I'll like to email in and action the script to get the results email >>> >>> >>> -- >>> Voytek >> >> Pacific.net >> Pacific.net From stan at hardwarefreak.com Fri Jul 13 15:07:04 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Jul 2012 07:07:04 -0500 Subject: [Dovecot] Maildir messages In-Reply-To: References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> Message-ID: <50000F68.3040003@hardwarefreak.com> On 7/13/2012 4:09 AM, Wojciech Puchar wrote: >> specifically from a filesystem IO perspective: >> >> 1. new mail delivery > > not much difference. maildir requires 3 (or is it 4?) metadata operations and a file write op mbox requires a single file append operation. >> 2. searching a mailbox folder > > if you do linear search then yes I'm referring to full text body search. In which case every single file in a maildir directory must be opened and searched in succession, serially, as Dovecot doesn't create a search thread for each maildir file and perform them in parallel across multiple cores. With mbox, you open a single file and search it. CPU/RAM bandwidth is many orders of magnitude faster and cheaper than disk IOPS. Thus mbox is faster at full text search than any other mailbox storage format, period. Full text indexes can help greatly with both formats, but often the indexes are old, and a full search is necessary, making mbox much faster. >> 3. mass deletion of emails from one mailbox folder >> > 3 is not true. It most certainly is true. You need to read up on how email deletes are performed on mbox files, or mdbox files for that matter. >> maildir is more IO efficient when reading and deleting individual emails. >> > and making backups. Wrong again. Streaming a single file to D2D or tape is much faster than random reading hundreds or thousands of maildir files. > deleting from maildir means just delete a file, not shuffle data. For a single email delete operation maildir is faster, as it requires a single metadata IO. When deleting many emails, say hundreds to thousands, as in deleting a very large folder, mbox is *much* faster. This is because CPU/mem are many orders of magnitude faster than disk, and deleting hundreds or thousands of maildir files requires hundreds or thousands of random metadata IOPS to the filesystem directory. When Qmail hit the scene with maildir format, everyone loved it. That is, until their mailbox counts skyrocketed, and their systems slow to a crawl because their disk arrays simply couldn't keep up with all the IOPS. > Everyone needs to make backups while it is unlinkely that anyone like > every backup to be effectively full backup. See: rdiff-backup, et al >> mbox puts the load on the mail server application and on memory. > > and on I/O too - often quite a bit Sure, if it's a busy server. But the IOPS load will always be much less than maildir given the same workload. >> maildir puts the load on the IO subsysetem. Which is precisely why Timo >> created the mdbox mail storage format, attempting to get the best of >> both worlds. > And this is great idea and actually works :) Yep. > mbox may make sense for archive storage. you create archive folder once > and never modify anything Many of us still use mbox for IMAP and POP user accounts, and it still works great. And many maildir converts switched back to mbox when the storage hardware required to satisfy their ever increasing maildir IOPS load began draining their entire IT budgets. mbox is a pretty smart email storage format especially given its age. It can do more with lesser storage hardware. Many simply don't give it the credit it deserves. -- Stan From CMarcus at Media-Brokers.com Fri Jul 13 15:19:15 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 13 Jul 2012 08:19:15 -0400 Subject: [Dovecot] Maildir messages In-Reply-To: <50000F68.3040003@hardwarefreak.com> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> Message-ID: <50001243.7090005@Media-Brokers.com> On 2012-07-13 8:07 AM, Stan Hoeppner wrote: > mbox is a pretty smart email storage format especially given its age. > It can do more with lesser storage hardware. Many simply don't give it > the credit it deserves. The ideal is a hybrid - maildir for new mail, mbox (or mdbox) for longer term storage... -- Best regards, Charles From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 15:28:36 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 14:28:36 +0200 (CEST) Subject: [Dovecot] Maildir messages In-Reply-To: <50000F68.3040003@hardwarefreak.com> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> Message-ID: >> >> not much difference. > > maildir requires 3 (or is it 4?) metadata operations and a file write op > your remarks are mostly true, except concentrates too much on uncommon situations and uncommon strange case of probably tens of thousands of mail accounts that must be very limited. >From my practice mail accounts are at least 1GB large, often 10GB. more often than not other user data is on the same disk. i've never had more than 50 users on single disk set (which is always 2 disks in mirror), And I/O was never ever a problem except first index build etc. in dovecot 2 you can do this in controller way using doveadm. Mail I/O doesn't make that much part of I/O for me, still i found a bit speedup by moving dovecot indexes to SSD if it exist. while disks get larger user data amount get larger too. Today tape backups are hardly used because it is plain uneconomic today. most do backups by copying differences to other disk, preferably in other place. Adventages of pure Maildir format outweights everything else as you have one file -> one mail. mdbox is something in between. From anmeyer at anup.de Fri Jul 13 15:29:31 2012 From: anmeyer at anup.de (anmeyer at anup.de) Date: Fri, 13 Jul 2012 14:29:31 +0200 Subject: [Dovecot] Maildir messages In-Reply-To: <50001243.7090005@Media-Brokers.com> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> <50001243.7090005@Media-Brokers.com> Message-ID: <22d83f6fbc7b8337610864f07813f7cc@anup.de> Am 13.07.2012 14:19, schrieb Charles Marcus: > On 2012-07-13 8:07 AM, Stan Hoeppner wrote: >> mbox is a pretty smart email storage format especially given its >> age. >> It can do more with lesser storage hardware. Many simply don't give >> it >> the credit it deserves. > > The ideal is a hybrid - maildir for new mail, mbox (or mdbox) for > longer term storage... may I ask how you do archivate maildirs? With mbox I used mhonarc but with maildir I do not really have a soluton yet. Andreas From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 15:31:34 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 14:31:34 +0200 (CEST) Subject: [Dovecot] Maildir messages In-Reply-To: <50001243.7090005@Media-Brokers.com> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> <50001243.7090005@Media-Brokers.com> Message-ID: >> the credit it deserves. > > The ideal is a hybrid - maildir for new mail, mbox (or mdbox) for longer term > storage... True. Not did this as never got I/O problems because of mail. For larger installations in may be adventageous to use mbox based archives on RAID5 (this would be mostly write-once) and RAID1 for rest. From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 15:33:34 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 14:33:34 +0200 (CEST) Subject: [Dovecot] Maildir messages In-Reply-To: <22d83f6fbc7b8337610864f07813f7cc@anup.de> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> <50001243.7090005@Media-Brokers.com> <22d83f6fbc7b8337610864f07813f7cc@anup.de> Message-ID: >> longer term storage... > > may I ask how you do archivate maildirs? With mbox I used mhonarc but > with maildir I do not really have a soluton yet. mhonarc is mail to HTML converter. We meant just converting format to mbox. From lists at necoro.eu Fri Jul 13 15:39:15 2012 From: lists at necoro.eu (=?ISO-8859-15?Q?Ren=E9_Neumann?=) Date: Fri, 13 Jul 2012 14:39:15 +0200 Subject: [Dovecot] imapc: Restrict the number of connections Message-ID: <500016F3.8060805@necoro.eu> Hi, I have managed to get an imapc connection to another server and bind its contents to only one account (via global ACL). The namespace itself is created as public, though it does not seem to make a difference for this use case (i.e., it does not matter, whether its type is 'public', 'shared', or 'private'). namespace { hidden = no list = children location = imapc: prefix = Gemeinsam. separator = . subscriptions = no type = public } One problem remains: Though there is only one account that is allowed to use this imapc-connection, each imap-process opens a connection to the imapc'd server. Is this caused directly by the imapc_*-definitions in the config? Or is there a way of restricting this to only the account which actually uses it. Used dovecot version: 2.1.8 Thanks, Ren? From eray.aslan at caf.com.tr Fri Jul 13 16:38:00 2012 From: eray.aslan at caf.com.tr (Eray Aslan) Date: Fri, 13 Jul 2012 16:38:00 +0300 Subject: [Dovecot] managesieve segfault with gold linker In-Reply-To: <4FFFF01A.9020402@rename-it.nl> References: <4FFF3326.9060802@caf.com.tr> <4FFFF01A.9020402@rename-it.nl> Message-ID: <500024B8.2020500@caf.com.tr> On 07/13/2012 12:53 PM, Stephan Bosch wrote: > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/rev/6ceeb6421231 Still no go with binutils-2.22. make check fails with the same error message. Telnetting and issuing checkscript gives not a segfault but some parser error. > I can't reproduce this (on Debian stable). How exactly did you configure > dovecot sources to use ld.gold? I just installed the binutils-gold > package, but I get the impression that ld.gold may not actually be used. Make sure /usr/bin/ld is a symlink to /usr/bin/ld.gold and not to ld.bfd - which it should if you installed binutils-gold. > Also, perhaps this is caused by a version difference (mine is GNU gold > (GNU Binutils for Debian 2.20.1-system.20100303) 1.9). I can't check binutils-2.20 easily. I did check linking with gold with binutils-2.21, binutils-2.22 and binutils-2.22.52.0.4. All fail. > I get the impression that this is an LD bug, but I do like to be sure. Aye, that's my impression as well. I'll try to find out more but I will be on vacation for a few days so it might not be quick. In the meantime, if you make any headway, please let me know. Thank you. -- Eray Aslan From andrzej.filip at gmail.com Fri Jul 13 17:15:43 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Fri, 13 Jul 2012 16:15:43 +0200 Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: References: Message-ID: <50002D8F.2000506@gmail.com> On 07/13/2012 11:14 AM, Wojciech Puchar wrote: >> anyone has any tips what's best way to execute a script via email ? > > use procmail - it's easy and actually works. you can do basically > everything with procmail. > > This question is off topic, dovecot is not SMTP agent. > >> I have a wget script that fetches some pages, and, emails me the output, >> there is no paramters passed, it's all in the script >> I'll like to email in and action the script to get the results email >> > > use procmail and possibly formail to execute anything and optionally > reply the result > > man procmailrc > man procmailex > man formail > > make sure procmail is actually executed on your mail system. depends > of your MTA. Are you willing to recommend procmail without any warnings/disclaimers whatsoever? Stable release 3.22 / September 10, 2001 [...] Procmail is widely used on Unix -based systems and stable, but no longer maintained; users who wish to use a maintained program are advised to use an alternative MDA, From stan at hardwarefreak.com Fri Jul 13 19:34:14 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 13 Jul 2012 11:34:14 -0500 Subject: [Dovecot] Maildir messages In-Reply-To: References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> Message-ID: <50004E06.7000101@hardwarefreak.com> On 7/13/2012 7:28 AM, Wojciech Puchar wrote: >>> >>> not much difference. >> >> maildir requires 3 (or is it 4?) metadata operations and a file write op >> > your remarks are mostly true, except concentrates too much on uncommon > situations and uncommon strange case of probably tens of thousands of > mail accounts that must be very limited. There are many folks on this list who ran out of IOPS due to maildir at one time or another. Nowhere near tens of thousands of accounts were not required. To solve the problem, they either purchased more spindles, added RAID cache, or converted to mbox, or mdbox, depending on when they ran into the problem (mdbox is very young). > From my practice mail accounts are at least 1GB large, often 10GB. more > often than not other user data is on the same disk. Typical SMB setup it seems. > i've never had more than 50 users on single disk set (which is always 2 > disks in mirror), And I/O was never ever a problem except first index > build etc. in dovecot 2 you can do this in controller way using doveadm. > > Mail I/O doesn't make that much part of I/O for me, still i found a bit > speedup by moving dovecot indexes to SSD if it exist. With only 50 users it's hard to saturate one disk with a mail workload. > while disks get larger user data amount get larger too. > > Today tape backups are hardly used because it is plain uneconomic today. Tape is far more cost effective than a GbE MAN link to a hot site. > most do backups by copying differences to other disk, preferably in > other place. Again, those links are usually more costly than off site tape rotation services, and the backup window much larger. > Adventages of pure Maildir format outweights everything else as you have > one file -> one mail. While that is maildir's greatest strength, it is also it's greatest weakness. -- Stan From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 21:45:22 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 20:45:22 +0200 (CEST) Subject: [Dovecot] Maildir messages In-Reply-To: <50004E06.7000101@hardwarefreak.com> References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> <50004E06.7000101@hardwarefreak.com> Message-ID: >> mail accounts that must be very limited. > > There are many folks on this list who ran out of IOPS due to maildir at > one time or another. Nowhere near tens of thousands of accounts were > not required. To solve the problem, they either purchased more did you actually monitored WHAT part of job generates I/O and how? Can you post details. On just one simple - how many mailboxes per disk head - what filesystem used - what OS - how data is laid out. > spindles, added RAID cache, or converted to mbox, or mdbox, depending on > when they ran into the problem (mdbox is very young). > >> From my practice mail accounts are at least 1GB large, often 10GB. more >> often than not other user data is on the same disk. > > Typical SMB setup it seems. True. More - i always try to mix services. No separate "file servers", "mail servers", "whatever" servers but all on same. On large install divide by groups of people, not services. This way i really cannot get to high IOPS. >> >> Mail I/O doesn't make that much part of I/O for me, still i found a bit >> speedup by moving dovecot indexes to SSD if it exist. > > With only 50 users it's hard to saturate one disk with a mail workload. True! That is my point. With user taking few GBs of mail space (few tens not uncommon), as much or more "documents" or other data, you just will never have more than 50 of then on one disk. More users=more disks, same (small) amount of I/O per disk head. > >> while disks get larger user data amount get larger too. >> >> Today tape backups are hardly used because it is plain uneconomic today. > > Tape is far more cost effective than a GbE MAN link to a hot site. I really cannot imagine a scale of operation that would require gigabit link to do backups over rsync that only moves changes! It would be like 10000 users on like 100 disks. With that site i would think of tape. Of course - no I/O problem to backup such 100 disk setup to tape. > Again, those links are usually more costly than off site tape rotation > services, and the backup window much larger. my practice shows exactly opposite case. If you would be right i would already use tapes or even pseudo-tapes== few cheapers hard drives connected over eSATA. > >> Adventages of pure Maildir format outweights everything else as you have >> one file -> one mail. > > While that is maildir's greatest strength, it is also it's greatest > weakness. For my needs strength is great and weakness is small. mdbox may help in your cases. Seems like you do same ill service like 1000 (or more) mail accounts per hard disks. My point was always "never delete old data, disks are cheap". From wojtek at wojtek.tensor.gdynia.pl Fri Jul 13 21:46:59 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 13 Jul 2012 20:46:59 +0200 (CEST) Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: <50002D8F.2000506@gmail.com> References: <50002D8F.2000506@gmail.com> Message-ID: > Stable release > 3.22 / > September 10, 2001 > [...] > Procmail is widely used on Unix > -based systems and stable, but no > longer maintained; users who wish to use a maintained > program are advised to use an alternative MDA, > can you explain what is wrong in using greatly working program just because there are no new versions? What kind of external "maintenance" you need for it? Sorry but your warning is a pure nonsense. procmail is mature software and that's adventage. From h.reindl at thelounge.net Fri Jul 13 22:05:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 13 Jul 2012 21:05:03 +0200 Subject: [Dovecot] Maildir messages In-Reply-To: References: <4FFE6AE3.2070003@5sg.eu> <4FFF70AA.3080103@hardwarefreak.com> <50000F68.3040003@hardwarefreak.com> <50004E06.7000101@hardwarefreak.com> Message-ID: <5000715F.80902@thelounge.net> Am 13.07.2012 20:45, schrieb Wojciech Puchar: >> Typical SMB setup it seems. > > True. More - i always try to mix services. No separate "file servers", "mail servers", "whatever" servers but all > on same. On large install divide by groups of people, not services. > > This way i really cannot get to high IOPS "cannot" is plain wrong, you are satisfied only by luck > Seems like you do same ill service like 1000 (or more) mail accounts per hard disks there is nothing ill http://h18000.www1.hp.com/products/quickspecs/13551_na/13551_na.pdf 1 Gb Ethernet / iSCSI -> 17200 Random IOPS with RAID10 1 Gb Ethernet / iSCSI -> 9400 Random IOPS with RAID5 1 Gb Ethernet / iSCSI -> 5600 Random IOPS with RAID6 and here ware are at a point where your local disks are outperformed because they give you typically 100-300 IOPS per disk which you refused over a complete thread believing you localSATA is the best thing whichs exists and yes, the dedicated storage-controllers and caches of a SAN storage are what is outperforming you at real load -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Fri Jul 13 22:06:52 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 13 Jul 2012 21:06:52 +0200 Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: References: <50002D8F.2000506@gmail.com> Message-ID: <500071CC.7070706@thelounge.net> Am 13.07.2012 20:46, schrieb Wojciech Puchar: >> Stable release >> 3.22 / >> September 10, 2001 >> [...] >> Procmail is widely used on Unix >> -based systems and stable, but no >> longer maintained; users who wish to use a maintained >> program are advised to use an alternative MDA, >> > can you explain what is wrong in using greatly working program just because there are no new versions? > > What kind of external "maintenance" you need for it? > Sorry but your warning is a pure nonsense. procmail is mature software and that's adventage http://www.cvedetails.com/product/392/Procmail-Procmail.html?vendor_id=225 hopefully there will be not found any new security problem now withut get fixed -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From andrzej.filip at gmail.com Sat Jul 14 00:38:00 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Fri, 13 Jul 2012 23:38:00 +0200 Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: <500071CC.7070706@thelounge.net> References: <50002D8F.2000506@gmail.com> <500071CC.7070706@thelounge.net> Message-ID: <50009538.50404@gmail.com> On 07/13/2012 09:06 PM, Reindl Harald wrote: > > Am 13.07.2012 20:46, schrieb Wojciech Puchar: >>> Stable release >>> 3.22 / >>> September 10, 2001 >>> [...] >>> Procmail is widely used on Unix >>> -based systems and stable, but no >>> longer maintained; users who wish to use a maintained >>> program are advised to use an alternative MDA, >>> >> can you explain what is wrong in using greatly working program just because there are no new versions? >> >> What kind of external "maintenance" you need for it? >> Sorry but your warning is a pure nonsense. procmail is mature software and that's adventage > http://www.cvedetails.com/product/392/Procmail-Procmail.html?vendor_id=225 > > hopefully there will be not found any new > security problem now withut get fixed 1) IMHO Stable stream of new versions usually indicate maintenance group ready to fix would be vulnerabilities. 2) I miss LMTP support in procmail. AFAIR it has been supported in some versions but current Debian version no longer mentions it in the man page. 10 years old version is something that IMHO must be mentioned when recommending procmail. For some people it is not important, for some people it is important information/hint. From nikolai at net24.co.nz Sat Jul 14 11:29:42 2012 From: nikolai at net24.co.nz (Nikolai Schupbach) Date: Sat, 14 Jul 2012 20:29:42 +1200 Subject: [Dovecot] Migrating with dsync and INBOX.INBOX namespace issue Message-ID: <663971B2-48A0-4C5B-9555-05D1AFFF9C5E@net24.co.nz> I'm working on migrating IMAP users from Mirapoint Message Servers to Dovecot servers. The Mirapoint IMAP server uses INBOX as a namespace for private mailboxes. I created the same namespace in Dovecot, but when I migrate mailboxes dsync seems to ignore/miss the namespace on Mirapoint and I end up with all mailboxes under a INBOX.INBOX hierarchy. Mirapoint Source Server ----------------------- * OK server.name Mirapoint IMAP4 3.8.3-GA server ready 1 LOGIN test at test.com test 1 OK User logged in 2 NAMESPACE * NAMESPACE (("INBOX." ".")) (("user." ".")) (("" ".")) 2 OK Completed 3 LIST "" * * LIST () "." "INBOX" * LIST () "." "INBOX.Draft" * LIST () "." "INBOX.Sent" * LIST () "." "INBOX.Trash" 3 OK Completed Running Dsync ------------- # doveadm -v -o imapc_host=server.name imapc_user=test at test.com -o imapc_password=test backup -R -u test at test.com imapc: dsync(test at test.com): Info: INBOX.INBOX.Sent: only in source (guid=7d3c7eaa71cdf47ee8a1192687cda8cd) dsync(test at test.com): Info: INBOX.INBOX.Draft: only in source (guid=a6a03ff3d6ec9066b46013f4affb109a) dsync(test at test.com): Info: INBOX.INBOX.Trash: only in source (guid=b36f688eae08506001091708006abe5a) dsync(test at test.com): Info: INBOX: only in source (guid=c92f64f79f0d1ed01e6d5b314f04886c) Dovecot Destination Server Post Dsync ------------------------------------- * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. 1 LOGIN test at test.com test 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE QUOTA ACL RIGHTS=texk] Logged in 2 NAMESPACE * NAMESPACE (("INBOX." ".")) NIL NIL 2 OK Namespace completed. 3 LIST "" * * LIST (\HasChildren) "." "INBOX" * LIST (\HasNoChildren) "." "INBOX.INBOX.Sent" * LIST (\HasNoChildren) "." "INBOX.INBOX.Trash" * LIST (\HasNoChildren) "." "INBOX.INBOX.Draft" 3 OK List completed. I noticed that Mirapoint IMAP also advertises a third empty namespace and I was wondering if this was causing it, but even if I explicitly set -n "INBOX." I still have the same issue. Below is my doveconf -n output: # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-RELEASE-p3 amd64 auth_cache_negative_ttl = 0 auth_verbose = yes default_vsz_limit = 512 M deliver_log_format = subject=%s msgid=%m: %$ disable_plaintext_auth = no imapc_features = rfc822.size mail_gid = vmail mail_home = /mailstore/domains/%d/%n mail_location = mdbox:~/mdbox mail_plugins = quota acl mail_log notify mail_temp_dir = /var/tmp mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave spamtest namespace { inbox = yes location = prefix = INBOX. separator = . } passdb { args = /usr/local/etc/dovecot/dovecot-ldap-passdb.conf driver = ldap } plugin { acl = vfile:/usr/local/etc/dovecot/global-acls mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::file:%h/.dovecot-quota quota_warning = storage=95%% quota-warning 95 %u sieve = ~/.dovecot.sieve sieve_extensions = +spamtest sieve_max_redirects = 32 sieve_spamtest_max_value = 10 sieve_spamtest_status_header = X-Spam-Score sieve_spamtest_status_type = score } service auth-worker { user = $default_internal_user } service auth { client_limit = 5420 unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = $default_internal_user } service imap-login { process_min_avail = 12 service_count = 0 vsz_limit = 512 M } service imap { process_limit = 4096 vsz_limit = 512 M } service lmtp { process_min_avail = 12 user = vmail } service pop3-login { process_min_avail = 12 service_count = 0 vsz_limit = 512 M } service pop3 { process_limit = 1024 vsz_limit = 512 M } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf driver = ldap } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 50 mail_plugins = quota acl mail_log notify imap_quota imap_acl } protocol lmtp { mail_plugins = quota acl mail_log notify sieve } Any help would be much appreciated. Regards, Nikolai. From wojtek at wojtek.tensor.gdynia.pl Sat Jul 14 20:48:07 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sat, 14 Jul 2012 19:48:07 +0200 (CEST) Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: <500071CC.7070706@thelounge.net> References: <50002D8F.2000506@gmail.com> <500071CC.7070706@thelounge.net> Message-ID: >> can you explain what is wrong in using greatly working program just because there are no new versions? >> >> What kind of external "maintenance" you need for it? >> Sorry but your warning is a pure nonsense. procmail is mature software and that's adventage > > http://www.cvedetails.com/product/392/Procmail-Procmail.html?vendor_id=225 > > hopefully there will be not found any new > security problem now withut get fixed > > so hack my mail. i use procmail heavily. i'm waiting From wojtek at wojtek.tensor.gdynia.pl Sat Jul 14 20:49:06 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sat, 14 Jul 2012 19:49:06 +0200 (CEST) Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: <50009538.50404@gmail.com> References: <50002D8F.2000506@gmail.com> <500071CC.7070706@thelounge.net> <50009538.50404@gmail.com> Message-ID: > versions but current Debian version no longer mentions it in the man page. > > 10 years old version is something that IMHO must be mentioned when > recommending procmail. For some people it is not important, for some > people it is important information/hint. > so you've mentioned. that's ok. but don't say it is bad. not everyone in the world choose software based on age. From h.reindl at thelounge.net Sat Jul 14 21:16:47 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 14 Jul 2012 20:16:47 +0200 Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: References: <50002D8F.2000506@gmail.com> <500071CC.7070706@thelounge.net> Message-ID: <5001B78F.2060404@thelounge.net> Am 14.07.2012 19:48, schrieb Wojciech Puchar: >>> can you explain what is wrong in using greatly working program just because there are no new versions? >>> >>> What kind of external "maintenance" you need for it? >>> Sorry but your warning is a pure nonsense. procmail is mature software and that's adventage >> >> http://www.cvedetails.com/product/392/Procmail-Procmail.html?vendor_id=225 >> >> hopefully there will be not found any new >> security problem now withut get fixed >> >> > so hack my mail. i use procmail heavily. i'm waiting would you stupid boy stop your permanently trolling on this list? what is your problem? that you saw the IOPS of SAN storages from me after your embarrassing discussion and now searching some other reason to piss around? i only pointed out that your "pure nonsense" statement is naive and if you can not stand criticism be quit at all -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From ben at morrow.me.uk Sat Jul 14 21:17:05 2012 From: ben at morrow.me.uk (Ben Morrow) Date: Sat, 14 Jul 2012 19:17:05 +0100 Subject: [Dovecot] [PATCH] Interop problem with Cyrus SASL and GSSAPI Message-ID: <20120714181704.GB72986@anubis.morrow.me.uk> [I sent this a while ago, but it seems not to have made it to the list. I'm resending it having subscribed first; I apologise if anyone get it twice.] I have been trying to get a Postfix mail server using Dovecot SASL to accept GSSAPI AUTH from another Postfix server using Cyrus SASL, and I believe I have found a couple of bugs in Dovecot's GSSAPI implementation. The first problem is that, because of the way the client invokes libsasl, it sends a GSSAPI request which does not ask for mutual authentication. This means that on the server gss_accept_sec_context returns GSS_S_COMPLETE with a zero-length output token. Dovecot currently sends this to the client as a zero-length continuation response, but this is incorrect according to RFC 4752: what it ought to do instead is proceed straight to the security layer negotiations, and send a gss_wrap packet. The second is that Cyrus sends an empty authz identity; that is, the security layer negotiation packet, when gss_unwrapped, is exactly 4 bytes long. Dovecot objects to this, but in RFC 4422 this is explicitly allowed, and means the authz identity is identical to the authn identity. I believe the attached patches (for the 1.2 and 2.1 branches) fix the problem, though I'm not entirely sure if the difference between the p_strndup in mech_gssapi_unwrap and the t_strndup in get_display_name is important. Ben -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot12-sasl-gssapi.patch Type: text/x-diff Size: 2859 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: dovecot21-sasl-gssapi.patch Type: text/x-diff Size: 2910 bytes Desc: not available URL: From wojtek at wojtek.tensor.gdynia.pl Sat Jul 14 21:28:26 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Sat, 14 Jul 2012 20:28:26 +0200 (CEST) Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: <5001B78F.2060404@thelounge.net> References: <50002D8F.2000506@gmail.com> <500071CC.7070706@thelounge.net> <5001B78F.2060404@thelounge.net> Message-ID: >> so hack my mail. i use procmail heavily. i'm waiting > > would you stupid boy stop your permanently trolling on this list? first i would need to start. From h.reindl at thelounge.net Sat Jul 14 21:30:19 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 14 Jul 2012 20:30:19 +0200 Subject: [Dovecot] ot: execute a script via email? [procmail] In-Reply-To: References: <50002D8F.2000506@gmail.com> <500071CC.7070706@thelounge.net> <5001B78F.2060404@thelounge.net> Message-ID: <5001BABB.9060408@thelounge.net> Am 14.07.2012 20:28, schrieb Wojciech Puchar: >>> so hack my mail. i use procmail heavily. i'm waiting >> >> would you stupid boy stop your permanently trolling on this list? > first i would need to start you even quoted it P.S.: youd o not need to reply-all if answering to members of a mailing-list, that is why "repyl to list" exists in thunderbird and other clients -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From michael at think-for-yourself.org Sun Jul 15 06:34:14 2012 From: michael at think-for-yourself.org (Michael Wessel) Date: Sat, 14 Jul 2012 20:34:14 -0700 Subject: [Dovecot] expire-tool stops on first box with nothing to expire in it Message-ID: <50023A36.4070705@think-for-yourself.org> Hi, I'm running dovecot 1.2.11 on Centos and have the expire plugin enabled. I've had this in place for some time but until now never actually ran the expire-tool. I'm now trying to put this to use. The expire plugin as such is working fine and also the tool is working fine - up to a point. When I run it with --test it goes through a bunch of boxes and lists the messages it would expunge and the new date being logged at the end. As soon as it hits the first box that has nothing in it that's expired however, it stops and goes no further. As an example here are the last two lines of the output when I just ran it: Info: user at domain.com/Trash: timestamp 1342320464 (Sat Jul 14 19:47:44 2012) -> 1342937319 (Sat Jul 21 23:08:39 2012) Info: nextuser at domain.com/Trash: stop, expire time in future: Sat Jul 14 20:41:49 2012 If I run it again after the indicated expire time it will now also go through the nextuser at domain.com but then again stop as soon as it encounters another user with nothing to expire. As far as I can tell the same happens when I actually run the tool. Harder to tell of course as there is no output, but I can tell that not all boxes are expunged as they should be. Any ideas why this would be? Any way to get debug logging on this? Michael From robert at schetterer.org Sun Jul 15 09:14:49 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 15 Jul 2012 08:14:49 +0200 Subject: [Dovecot] expire-tool stops on first box with nothing to expire in it In-Reply-To: <50023A36.4070705@think-for-yourself.org> References: <50023A36.4070705@think-for-yourself.org> Message-ID: <50025FD9.6000701@schetterer.org> Am 15.07.2012 05:34, schrieb Michael Wessel: > Hi, > > I'm running dovecot 1.2.11 on Centos and have the expire plugin enabled. > I've had this in place for some time but until now never actually ran > the expire-tool. I'm now trying to put this to use. > > The expire plugin as such is working fine and also the tool is working > fine - up to a point. > > When I run it with --test it goes through a bunch of boxes and lists the > messages it would expunge and the new date being logged at the end. As > soon as it hits the first box that has nothing in it that's expired > however, it stops and goes no further. As an example here are the last > two lines of the output when I just ran it: > > Info: user at domain.com/Trash: timestamp 1342320464 (Sat Jul 14 19:47:44 > 2012) -> 1342937319 (Sat Jul 21 23:08:39 2012) > Info: nextuser at domain.com/Trash: stop, expire time in future: Sat Jul 14 > 20:41:49 2012 > > If I run it again after the indicated expire time it will now also go > through the nextuser at domain.com but then again stop as soon as it > encounters another user with nothing to expire. > > As far as I can tell the same happens when I actually run the tool. > Harder to tell of course as there is no output, but I can tell that not > all boxes are expunged as they should be. > > Any ideas why this would be? Any way to get debug logging on this? > > Michael 1.2.11 is outdated you should upgrade 2.1.8 and try again however 1.2.17 is latest patch level for 1.2 -- Best Regards MfG Robert Schetterer From nick.z.edwards at gmail.com Sun Jul 15 10:11:14 2012 From: nick.z.edwards at gmail.com (Nick Edwards) Date: Sun, 15 Jul 2012 17:11:14 +1000 Subject: [Dovecot] bcypt availability In-Reply-To: References: Message-ID: bump On 7/12/12, Nick Edwards wrote: > Dear Timo, > > Do you intend to introduce bcrypt into the built in password schemes? > > In lew of all these hacks lately many larger companies appear moving > this way, we are looking at it too, but dovecot will then be the > weakest link in the database security. > > So, are you planning on this and if so what sort of timeframe / > version would you expect it to be in beta ? > > Nik > From kamath at moltingpenguin.com Sun Jul 15 11:50:23 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Sun, 15 Jul 2012 01:50:23 -0700 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. Message-ID: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> Hi all, So, it's been a while. I'm just ramping back up with Dovecot after a hiatus of a few years. I've been using it fine, complete with Sieve, for about a year, with little or no issues. I finally got around to setting up a second account, to start figuring out how I want to do some plus address filtering. In a nutshell, I want to put mail for foo+bar in foo's 'bar' folder. That's in the wiki. I did this: require ["fileinto", "envelope", "subaddress"]; if header :contains "subject" ["test"] { fileinto "Test"; } if envelope :detail "to" "test" { fileinto "test"; stop; } So, this fails miserably. No error message, nothing in the logs, no nothing. Yet, when I run sieve-test .dovecot.sieve /path/to/testmessage it works just fine. I even run sieve-test -e .dovecot.sieve /path/to/testmessage and it files it just fine into the folder. I tried adding some debug logging, but it looks like vnd.dovecot.debug isn't in my version. I searched the archives, and found a bit about adding (explicitly) the delimiter to the plugin section of 90-sieve.conf. So I did that. I'm running 2.0.13 of dovecot, and pigeonhole 0.2.3 on Solaris 10. Before I dive in and upgrade everything, am I missing something simple? The "test" bit works just fine (and in another account as well), so I think something's just not seeing the detail in subaddressing when mail is delivered (even though it works just fine with sieve-test). Thanks, Sean From lists at wildgooses.com Sun Jul 15 12:14:21 2012 From: lists at wildgooses.com (Ed W) Date: Sun, 15 Jul 2012 10:14:21 +0100 Subject: [Dovecot] bcypt availability In-Reply-To: References: Message-ID: <500289ED.3060705@wildgooses.com> > On 7/12/12, Nick Edwards wrote: >> Dear Timo, >> >> Do you intend to introduce bcrypt into the built in password schemes? >> >> In lew of all these hacks lately many larger companies appear moving >> this way, we are looking at it too, but dovecot will then be the >> weakest link in the database security. >> >> So, are you planning on this and if so what sort of timeframe / >> version would you expect it to be in beta ? >> >> Nik >> Interestingly, there doesn't seem to be so much difference between iterated sha-512 (sha512crypt) and bcrypt. Based on looking at latest john the ripper results (although I'm a bit confused because they don't seem to quote the baseline results using the normal default number of rounds?) So I think right now, many/most modern glibc are shipping with sha256/512crypt implementations (recently uclibc also added this). A small number ship with bcrypt (I have a patch for uclibc), which would mean that dovecot supported bcrypt out of the box. For everything else I guess you want a small application and use the checkpass dovecot method to do external checking? You could for example implement scrypt checking this way (although I think there is a risk of running out of server ram if you have many simultaneous logins..?) I previously thought I wanted bcrypt, but after some consideration I believe sha256/512crypt is likely sufficient for reasonable security Cheers Ed W From nikolai at net24.co.nz Sun Jul 15 12:39:28 2012 From: nikolai at net24.co.nz (Nikolai Schupbach) Date: Sun, 15 Jul 2012 21:39:28 +1200 Subject: [Dovecot] Migrating with dsync and INBOX.INBOX namespace issue In-Reply-To: <663971B2-48A0-4C5B-9555-05D1AFFF9C5E@net24.co.nz> References: <663971B2-48A0-4C5B-9555-05D1AFFF9C5E@net24.co.nz> Message-ID: <2D219064-4E74-4FAD-A3E0-CE609F4349B7@net24.co.nz> Just a follow up to this - I used the imapsync perl script and found that it synced with the namespaces correctly: /usr/local/bin/imapsync --host1 source.server.name --user1 test at test.com --password1 MASKED --host2 dest.server.name --user2 test at test.com --password2 MASKED Host1 capability: IMAP4 IMAP4rev1 ACL QUOTA LITERAL+ NAMESPACE UIDPLUS IDLE AUTH=PLAIN UNSELECT Host2 capability: IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE QUOTA ACL RIGHTS=texk Host1 separator and prefix: [.][INBOX.] Host2 separator and prefix: [.][INBOX.] So there appears to be some problem with namespaces and dsync/imapc. On 14/07/2012, at 8:29 PM, Nikolai Schupbach wrote: > I'm working on migrating IMAP users from Mirapoint Message Servers to Dovecot servers. The Mirapoint IMAP server uses INBOX as a namespace for private mailboxes. I created the same namespace in Dovecot, but when I migrate mailboxes dsync seems to ignore/miss the namespace on Mirapoint and I end up with all mailboxes under a INBOX.INBOX hierarchy. > > Mirapoint Source Server > ----------------------- > * OK server.name Mirapoint IMAP4 3.8.3-GA server ready > 1 LOGIN test at test.com test > 1 OK User logged in > 2 NAMESPACE > * NAMESPACE (("INBOX." ".")) (("user." ".")) (("" ".")) > 2 OK Completed > 3 LIST "" * > * LIST () "." "INBOX" > * LIST () "." "INBOX.Draft" > * LIST () "." "INBOX.Sent" > * LIST () "." "INBOX.Trash" > 3 OK Completed > > Running Dsync > ------------- > # doveadm -v -o imapc_host=server.name imapc_user=test at test.com -o imapc_password=test backup -R -u test at test.com imapc: > dsync(test at test.com): Info: INBOX.INBOX.Sent: only in source (guid=7d3c7eaa71cdf47ee8a1192687cda8cd) > dsync(test at test.com): Info: INBOX.INBOX.Draft: only in source (guid=a6a03ff3d6ec9066b46013f4affb109a) > dsync(test at test.com): Info: INBOX.INBOX.Trash: only in source (guid=b36f688eae08506001091708006abe5a) > dsync(test at test.com): Info: INBOX: only in source (guid=c92f64f79f0d1ed01e6d5b314f04886c) > > Dovecot Destination Server Post Dsync > ------------------------------------- > * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. > 1 LOGIN test at test.com test > 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE QUOTA ACL RIGHTS=texk] Logged in > 2 NAMESPACE > * NAMESPACE (("INBOX." ".")) NIL NIL > 2 OK Namespace completed. > 3 LIST "" * > * LIST (\HasChildren) "." "INBOX" > * LIST (\HasNoChildren) "." "INBOX.INBOX.Sent" > * LIST (\HasNoChildren) "." "INBOX.INBOX.Trash" > * LIST (\HasNoChildren) "." "INBOX.INBOX.Draft" > 3 OK List completed. > > I noticed that Mirapoint IMAP also advertises a third empty namespace and I was wondering if this was causing it, but even if I explicitly set -n "INBOX." I still have the same issue. > > Below is my doveconf -n output: > > # 2.1.6: /usr/local/etc/dovecot/dovecot.conf > # OS: FreeBSD 9.0-RELEASE-p3 amd64 > auth_cache_negative_ttl = 0 > auth_verbose = yes > default_vsz_limit = 512 M > deliver_log_format = subject=%s msgid=%m: %$ > disable_plaintext_auth = no > imapc_features = rfc822.size > mail_gid = vmail > mail_home = /mailstore/domains/%d/%n > mail_location = mdbox:~/mdbox > mail_plugins = quota acl mail_log notify > mail_temp_dir = /var/tmp > mail_uid = vmail > managesieve_notify_capability = mailto > managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave spamtest > namespace { > inbox = yes > location = > prefix = INBOX. > separator = . > } > passdb { > args = /usr/local/etc/dovecot/dovecot-ldap-passdb.conf > driver = ldap > } > plugin { > acl = vfile:/usr/local/etc/dovecot/global-acls > mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename > mail_log_fields = uid box msgid size > quota = dict:User quota::file:%h/.dovecot-quota > quota_warning = storage=95%% quota-warning 95 %u > sieve = ~/.dovecot.sieve > sieve_extensions = +spamtest > sieve_max_redirects = 32 > sieve_spamtest_max_value = 10 > sieve_spamtest_status_header = X-Spam-Score > sieve_spamtest_status_type = score > } > service auth-worker { > user = $default_internal_user > } > service auth { > client_limit = 5420 > unix_listener auth-userdb { > group = vmail > mode = 0600 > user = vmail > } > user = $default_internal_user > } > service imap-login { > process_min_avail = 12 > service_count = 0 > vsz_limit = 512 M > } > service imap { > process_limit = 4096 > vsz_limit = 512 M > } > service lmtp { > process_min_avail = 12 > user = vmail > } > service pop3-login { > process_min_avail = 12 > service_count = 0 > vsz_limit = 512 M > } > service pop3 { > process_limit = 1024 > vsz_limit = 512 M > } > service quota-warning { > executable = script /usr/local/bin/quota-warning > unix_listener quota-warning { > user = vmail > } > user = vmail > } > ssl = no > userdb { > args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf > driver = ldap > } > verbose_proctitle = yes > protocol imap { > mail_max_userip_connections = 50 > mail_plugins = quota acl mail_log notify imap_quota imap_acl > } > protocol lmtp { > mail_plugins = quota acl mail_log notify sieve > } > > Any help would be much appreciated. > > Regards, Nikolai. > > From CMarcus at Media-Brokers.com Sun Jul 15 14:34:03 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 15 Jul 2012 07:34:03 -0400 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> Message-ID: <5002AAAB.6080702@Media-Brokers.com> On 2012-07-15 4:50 AM, Sean Kamath wrote: > Before I dive in and upgrade everything, am I missing something simple? dovecot -n output maybe? From dovecot at r.paypc.com Sun Jul 15 21:32:12 2012 From: dovecot at r.paypc.com (Robin) Date: Sun, 15 Jul 2012 11:32:12 -0700 Subject: [Dovecot] bcypt availability In-Reply-To: <500289ED.3060705@wildgooses.com> References: <500289ED.3060705@wildgooses.com> Message-ID: <50030CAC.9000302@r.paypc.com> On 7/15/2012 2:14 AM, Ed W wrote: > > Interestingly, there doesn't seem to be so much difference between > iterated sha-512 (sha512crypt) and bcrypt. Based on looking at latest > john the ripper results (although I'm a bit confused because they don't > seem to quote the baseline results using the normal default number of > rounds?) > > So I think right now, many/most modern glibc are shipping with > sha256/512crypt implementations (recently uclibc also added this). Indeed. What I have seen is a create deal of variation in the configuration (/etc/login.defs or your distro's equivalent) in terms of making use of such things. I don't see any added value to bcrypt over iterated SHA-512, really, and while I don't even pretend to claim I've looked at all distros, even "old-school" ones like Slackware have full support for it. I suspect many admins doubt this because of configurations that don't make use of the modern hashing functionality. Converting shadow files and/or login.defs would seem to be the bulk of the SysAdmin work to beef up the protection to bcrypt levels here. Remember to keep this in perspective though - as the nature of this "vulnerability" extends to the case where your shadow file's hashes have been cloned, meaning a root-compromise or local device clone/access was made of it, etc. =R= From tom at whyscream.net Sun Jul 15 23:27:52 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Sun, 15 Jul 2012 22:27:52 +0200 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <5002AAAB.6080702@Media-Brokers.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> Message-ID: <500327C8.60301@whyscream.net> On 15-07-12 13:34, Charles Marcus wrote: > On 2012-07-15 4:50 AM, Sean Kamath wrote: >> Before I dive in and upgrade everything, am I missing something simple? > > dovecot -n output maybe? Adding some log output of a message that, according to you, should have been delivered to folder Bar but ended up somewhere else, would be nice too. Be sure to include both logging of dovecot and the MTA talking to dovecot (you didn't mention which one). -- Tom From Nick at 5sg.eu Mon Jul 16 01:22:38 2012 From: Nick at 5sg.eu (Nick at 5sg.eu) Date: Sun, 15 Jul 2012 18:22:38 -0400 Subject: [Dovecot] Maildir messages Message-ID: <500342AE.9020801@5sg.eu> Hi Wojciech Thanks for your comeback.. > maybe - just as in article below > http://www.bishnet.net/tim/blog/tag/maildir-to-mbox/ > you like archive in such format I'm trying to switch dovecot to MBox, not just make archives. BTW: I use http://yergler.net/projects/one-off/maildir-to-mbox as it's much smaller and very simple to use to make archives. Does anyone else know the correct way to either switch the existing Maildir multiple file format to the single MBox file format. Failing that, though it will mean a LOT of work, how do I get dovecot to work in future in MBox single file format. I've entered 'mail_location = mbox:~/mail:INBOX=/%h/%u' in the 'dovecot.conf', but it still saves each message in a file of its own in the /new /cur /tmp directories ?? -------Original question -------- > Can someone guide me on changing the Maildir messages files > back to the single file format I used to have instead of the > dovecot.index /cur /tmp /new format my newer server currently > has. I'm running dovecot v1.2.9 postfix v2.7 on Ubuntu v10.4 Thanks, Nick. . . . . -- _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ . . . . . . . . ...---... (Stamp Out Spam and Save Our Servers) http://sosNow.ws -=[ do you know morse code ? ]=- . . . . . . . . _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ From kamath at moltingpenguin.com Mon Jul 16 01:24:29 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Sun, 15 Jul 2012 15:24:29 -0700 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> Message-ID: <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> On Jul 15, 2012, at 4:34 AM, Charles Marcus wrote: > On 2012-07-15 4:50 AM, Sean Kamath wrote: >> Before I dive in and upgrade everything, am I missing something simple? > > dovecot -n output maybe? Certainly (wasn't on the list of "things to put in a post" on the web site, so I missed it): # 2.0.13: /etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4u base_dir = /var/run/dovecot/ hostname = mail.enoent.net lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = X-Original-To lmtp_save_to_detail_mailbox = yes mail_location = sdbox:~/Mail:INDEX=/var/dovecot/indices/%u managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { driver = pam } plugin { recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_dir = ~/.sieve } protocols = imap sieve ssl_cert = References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <500327C8.60301@whyscream.net> Message-ID: <1DA9CEB1-4775-44EF-B256-FBF2F474B7A2@moltingpenguin.com> On Jul 15, 2012, at 1:27 PM, Tom Hendrikx wrote: > On 15-07-12 13:34, Charles Marcus wrote: >> On 2012-07-15 4:50 AM, Sean Kamath wrote: >>> Before I dive in and upgrade everything, am I missing something simple? >> >> dovecot -n output maybe? > > Adding some log output of a message that, according to you, should have > been delivered to folder Bar but ended up somewhere else, would be nice > too. > Be sure to include both logging of dovecot and the MTA talking to > dovecot (you didn't mention which one). Heh. I spent 1/2 an hour rerunning all the tests, gathering detailed logs, the whole nine yards and then, as an afterthought, added this: Here's the relevant portion of my sendmail.mc file for the MAILER definition of dovecot-lda: FEATURE(`local_procmail', `/usr/local/libexec/dovecot/dovecot-lda',`/usr/local/libexec/dovecot/dovecot-lda -d $u') MODIFY_MAILER_FLAGS(`LOCAL', `-f') MAILER(procmail) Sure enough, since there is no "X-Original-To" header in the messages, and I don't specify the -a option to dovecot-lda, there's failure. Does anyone happen to know the right way to get that full address through? The only note about anything remotely related to this is in the dovecot-lda which states that -d is not necessary with system users (all my accounts will be system users). Sean From rblayzor.bulk at inoc.net Mon Jul 16 02:31:29 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Sun, 15 Jul 2012 19:31:29 -0400 Subject: [Dovecot] Last login datetime on accounts Message-ID: I have searching for the best way to tell if a Dovecot user is idle by finding the last time they have logged in. Right now the best way I can tell is to do something like "touch" a file in the users homedir via a post login exec. This seems like a LOT of overhead, to accomplish such a simple task, especially if your doing a lot of dozens of logins per second. Is there an easier/better way? Is there a way to possibly tell Dovecot to refresh the mod time on an index file whenever the user successfully auths or to refresh a separate file in the homedir, ie: ~/.lastlogin (not ~/Maildir/.lastlogin as that would obviously cause some issues) Seems like something that would be far less overhead than calling a shell, execing another command, etc. I'm open to other ideas/alternatives if they are out there! TIA -- Robert Blayzor INOC, LLC rblayzor at inoc.net http://www.inoc.net/~rblayzor/ From dmalolepszy at optusnet.com.au Mon Jul 16 06:02:38 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Mon, 16 Jul 2012 13:02:38 +1000 Subject: [Dovecot] director: non standart ports at backends In-Reply-To: <213B51F00051AE48A9F0E112880177178F79F1@Delta.sc.local> References: <213B51F00051AE48A9F0E112880177178F79EF@Delta.sc.local> <213B51F00051AE48A9F0E112880177178F79F0@Delta.sc.local> <8568BABD-F72C-47B2-B9A4-4902410404C6@iki.fi> <213B51F00051AE48A9F0E112880177178F79F1@Delta.sc.local> Message-ID: <5003844E.30804@optusnet.com.au> Apologies for drudging up an old thread, but I was wondering if you experienced any issues with the way dovecot caches sql results using your discussed setup? I have used your thread as a guide to run dovecot backend on non standard ports. From what I understand Dovecot will auto index the cache using the %a variable (which is good), however the returned cache also contains the old username, which causes the dovecot proxy to rewrite the username prior to passing on the user/pass details to the backend. I have discussed the issue here: http://www.dovecot.org/list/dovecot/2012-July/067072.html On 12/06/12 17:29, ???????? ????????? ?????????? wrote: > Thanks, that worked! > > I ended up with: > password_query = select 'y' as proxy, \ > NULL AS password, \ > 'y' as nopassword, \ > case '%a' \ > when 110 then 2110 \ > when 995 then 2995 \ > when 143 then 2143 \ > when 993 then 2993 \ > when 24 then 224 \ > when 4190 then 24190 end \ > as port, \ > case '%a' \ > when 995 then 'any-cert' \ > when 993 then 'any-cert' end \ > as `ssl`; > > > > > -----Original Message----- > From: Timo Sirainen [mailto:tss at iki.fi] > Sent: Monday, June 11, 2012 11:48 PM > To: ???????? ????????? ?????????? > Cc: dovecot at dovecot.org > Subject: Re: [Dovecot] director: non standart ports at backends > > Looking at your old mails, you seem to be using passdb static for director, but userdb sql? So you could switch to: > > passdb { > driver = sql > args = /etc/dovecot/dovecot-sql.conf > } > > password_query = select 'y' as proxy, 'y' as nopassword, if('%a'=143, 1430, 9930) as port > > where you'd change the if() to something that handles %s=imap vs %s=pop3 vs %s=lmtp and %a=143 vs %a=993 vs %a=110 vs %a=995. Maybe a "case" statement would be less ugly. Or simply make it a real table in sql. Anyway, that's the basic idea. > > On 11.6.2012, at 15.39, ???????? ????????? ?????????? wrote: > >> thanks Timo, for you time >> but I still don't get it) >> should I return "port" with just "port_num1,port_num2" value or how? >> I've tried to google an example but with no success. >> >> -----Original Message----- >> From: Timo Sirainen [mailto:tss at iki.fi] >> Sent: Monday, June 11, 2012 11:01 PM >> To: ???????? ????????? ?????????? >> Cc: dovecot at dovecot.org >> Subject: Re: [Dovecot] director: non standart ports at backends >> >> On 11.6.2012, at 12.27, ???????? ????????? ?????????? wrote: >> >>> hello, >>> I'm trying to figure out how to proxy pop3 and pop3s that listens on non-standart ports at backends. >>> For example, pop3 is at 1110 and pop3s at 1995 (on backend side). >>> is it possible? >>> how should I separate this ports in director's config? >>> it's easy for one port: >>> for example lmtp - you just use passdb in protocol lmtp {} >> >> The passdb needs to return the "port" field. You can't use static passdb for this, since it has no conditionals and you can't do per-port configuration. Maybe use sqlite (simply to use it as a scripting engine - empty database) or checkpassword as your passdb. >> > > > From harri at afaics.de Mon Jul 16 08:20:15 2012 From: harri at afaics.de (Harald Dunkel) Date: Mon, 16 Jul 2012 07:20:15 +0200 Subject: [Dovecot] incremental backups of maildir via rsync Message-ID: <5003A48F.3050505@afaics.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi folks, I would like to do incremental backups of the users' maildir directories to a dedicated backup host (using "rsync -SHa" and rsnapshot). Problem is: Dovecot bloats the incremental backup by changing the file names again and again. Is there some way to tell Dovecot to keep meta information separate from the file names? Any helpful comment would be highly appreciated. Harri -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlADpIsACgkQUTlbRTxpHjdS+ACgkPRnzRD2jrcz09kQx9jj9j5v Y1UAniJdRa14hc53gLbxkuBPbx5V5BCh =6nvA -----END PGP SIGNATURE----- From gedalya at gedalya.net Mon Jul 16 08:32:35 2012 From: gedalya at gedalya.net (Gedalya) Date: Mon, 16 Jul 2012 01:32:35 -0400 Subject: [Dovecot] incremental backups of maildir via rsync In-Reply-To: <5003A48F.3050505@afaics.de> References: <5003A48F.3050505@afaics.de> Message-ID: <5003A773.5080308@gedalya.net> On 07/16/2012 01:20 AM, Harald Dunkel wrote: > Is there some way to tell Dovecot to keep meta information > separate from the file names? If yes, then it's no longer maildir. From wojtek at wojtek.tensor.gdynia.pl Mon Jul 16 09:20:18 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Mon, 16 Jul 2012 08:20:18 +0200 (CEST) Subject: [Dovecot] incremental backups of maildir via rsync In-Reply-To: <5003A48F.3050505@afaics.de> References: <5003A48F.3050505@afaics.de> Message-ID: > I would like to do incremental backups of the users' maildir > directories to a dedicated backup host (using "rsync -SHa" > and rsnapshot). dont use it. use rsync -b option. It is MUCH MUCH BETTER. i use it regularly. Split your rsync session if you have huge amount of users or rsync would be taking enormous amount of RAM and have larger chance to fail. > Problem is: Dovecot bloats the incremental backup by changing > the file names again and again. I didn't observe that except when moving from "new" to "cur" but --detect-renamed fixes it. > > Is there some way to tell Dovecot to keep meta information > separate from the file names? > it already do this. exclude dovecot indexes from backup PS. rsync backup is what i use regularly. Except none of my servers are purely mail server. From robert at schetterer.org Mon Jul 16 09:45:32 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Jul 2012 08:45:32 +0200 Subject: [Dovecot] Last login datetime on accounts In-Reply-To: References: Message-ID: <5003B88C.6020907@schetterer.org> Am 16.07.2012 01:31, schrieb Robert Blayzor: > I have searching for the best way to tell if a Dovecot user is idle by finding the last time they have logged in. > > Right now the best way I can tell is to do something like "touch" a file in the users homedir via a post login exec. > > This seems like a LOT of overhead, to accomplish such a simple task, especially if your doing a lot of dozens of logins per second. Is there an easier/better way? Is there a way to possibly tell Dovecot to refresh the mod time on an index file whenever the user successfully auths or to refresh a separate file in the homedir, ie: ~/.lastlogin (not ~/Maildir/.lastlogin as that would obviously cause some issues) > > Seems like something that would be far less overhead than calling a shell, execing another command, etc. > > I'm open to other ideas/alternatives if they are out there! > > TIA > i have running touch with 3000 users, i dont see much overhead, anyway its true ,its not very elegant, perhaps i.e you may write some daily cron bash find script looking about latest timestamp of files in .new with maildir which of cource is not the same as last_login script, but may good enough for you -- Best Regards MfG Robert Schetterer From robert at schetterer.org Mon Jul 16 09:46:36 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Jul 2012 08:46:36 +0200 Subject: [Dovecot] incremental backups of maildir via rsync In-Reply-To: <5003A48F.3050505@afaics.de> References: <5003A48F.3050505@afaics.de> Message-ID: <5003B8CC.6040005@schetterer.org> Am 16.07.2012 07:20, schrieb Harald Dunkel: > Hi folks, > > I would like to do incremental backups of the users' maildir > directories to a dedicated backup host (using "rsync -SHa" > and rsnapshot). > > Problem is: Dovecot bloats the incremental backup by changing > the file names again and again. > > Is there some way to tell Dovecot to keep meta information > separate from the file names? > > > Any helpful comment would be highly appreciated. > > Harri > dsync backup ,may the better the solution in your case -- Best Regards MfG Robert Schetterer From noel.butler at ausics.net Mon Jul 16 13:05:09 2012 From: noel.butler at ausics.net (Noel Butler) Date: Mon, 16 Jul 2012 20:05:09 +1000 Subject: [Dovecot] bcypt availability In-Reply-To: <50030CAC.9000302@r.paypc.com> References: <500289ED.3060705@wildgooses.com> <50030CAC.9000302@r.paypc.com> Message-ID: <1342433109.3843.4.camel@tardis> On Sun, 2012-07-15 at 11:32 -0700, Robin wrote: > Indeed. What I have seen is a create deal of variation in the > configuration (/etc/login.defs or your distro's equivalent) in terms of > making use of such things. > > I don't see any added value to bcrypt over iterated SHA-512, really, and bcrypt and scrypt are password hash's - they are designed to be slow md5/sha/sha2 are cryptographic hash's - they are designed to be fast However, if you keep your database secure, yes, this means using competent coders, then it matters little what method you use. Of course if you allow system access to users, the strongest is better than nothing. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From CMarcus at Media-Brokers.com Mon Jul 16 13:35:52 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 16 Jul 2012 06:35:52 -0400 Subject: [Dovecot] Maildir messages In-Reply-To: <500342AE.9020801@5sg.eu> References: <500342AE.9020801@5sg.eu> Message-ID: <5003EE88.4000402@Media-Brokers.com> On 2012-07-15 6:22 PM, Nick at 5sg.eu wrote: >> maybe - just as in article below >> http://www.bishnet.net/tim/blog/tag/maildir-to-mbox/ >> you like archive in such format > I'm trying to switch dovecot to MBox, not just make archives. > BTW: I use http://yergler.net/projects/one-off/maildir-to-mbox as > it's much smaller and very simple to use to make archives. Why do people insist on using $random_howto on the internet instead of the official documentation? > Does anyone else know the correct way to either switch the > existing Maildir multiple file format to the single MBox file > format. Failing that, though it will mean a LOT of work, how do I > get dovecot to work in future in MBox single file format. > I've entered 'mail_location = mbox:~/mail:INBOX=/%h/%u' in the > 'dovecot.conf', but it still saves each message in a file of its > own in the /new /cur /tmp directories ?? Maybe... http://wiki2.dovecot.org/Migration/MailFormat ? -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Jul 16 13:44:47 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 16 Jul 2012 06:44:47 -0400 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> Message-ID: <5003F09F.40908@Media-Brokers.com> On 2012-07-15 6:24 PM, Sean Kamath wrote: > On Jul 15, 2012, at 4:34 AM, Charles Marcus wrote: >> On 2012-07-15 4:50 AM, Sean Kamath wrote: >>> Before I dive in and upgrade everything, am I missing something >>> simple? >> dovecot -n output maybe > Certainly (wasn't on the list of "things to put in a post" on the web > site, so I missed it): Which website? http://dovecot.org/bugreport.html I have suggested that Timo put a short simple list of troubleshooting suggestions (ie, asking for dovecot -n output and basic non-verbose logs (unless/until asked for verbose logs)) in the welcome message when subscribing to the list (like postfix does)... but in all honesty, most people don't read those either. Regardless, providing configuration details and logs exhibiting the problem is one the most basic requirements when asking for help on any email support list I've ever been on... Not sure how anyone was supposed to guess that you were using sendmail either... And even when you replied to toms message suggesting you provide some logs, you still didn't... We'll be happy to try to help you, but we'll need your help in order to do that... -- Best regards, Charles From CMarcus at Media-Brokers.com Mon Jul 16 13:48:12 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 16 Jul 2012 06:48:12 -0400 Subject: [Dovecot] Last login datetime on accounts In-Reply-To: <5003B88C.6020907@schetterer.org> References: <5003B88C.6020907@schetterer.org> Message-ID: <5003F16C.8090204@Media-Brokers.com> On 2012-07-16 2:45 AM, Robert Schetterer wrote: > i have running touch with 3000 users, i dont see much overhead, anyway > its true ,its not very elegant, perhaps i.e you may write some daily > cron bash find script looking about latest timestamp of files in .new > with maildir > which of cource is not the same as last_login script, but may good > enough for you Why not a simple post-login script that updates your userDB? http://wiki2.dovecot.org/PostLoginScripting -- Best regards, Charles From robert at schetterer.org Mon Jul 16 14:01:20 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Jul 2012 13:01:20 +0200 Subject: [Dovecot] Last login datetime on accounts In-Reply-To: <5003F16C.8090204@Media-Brokers.com> References: <5003B88C.6020907@schetterer.org> <5003F16C.8090204@Media-Brokers.com> Message-ID: <5003F480.4030907@schetterer.org> Am 16.07.2012 12:48, schrieb Charles Marcus: > On 2012-07-16 2:45 AM, Robert Schetterer wrote: >> i have running touch with 3000 users, i dont see much overhead, anyway >> its true ,its not very elegant, perhaps i.e you may write some daily >> cron bash find script looking about latest timestamp of files in .new >> with maildir >> which of cource is not the same as last_login script, but may good >> enough for you > > Why not a simple post-login script that updates your userDB? > > http://wiki2.dovecot.org/PostLoginScripting > i may fail, but in the orig question PostLoginScripting is allready used but with touch a last_login file so update some db with same mech may not much better i thought some other way then PostLoginScripting was searched for -- Best Regards MfG Robert Schetterer From rrosenfeld at netcologne.de Mon Jul 16 15:22:22 2012 From: rrosenfeld at netcologne.de (Roland Rosenfeld) Date: Mon, 16 Jul 2012 14:22:22 +0200 Subject: [Dovecot] ACL with MySQL dict => full table scans Message-ID: <20120716122222.GA23019@sys-241.netcologne.de> Hi! I'm just installing ACL support on 2.1.6. When a user changes these ACLs, this results in a full table scan on the dictionary, that tells dovecot who shares with whom. Here's what queries I see when I run $ doveadm acl delete -u user1 shared/user2/folder user=user1 1: SELECT dummy,to_user,from_user FROM imap_acl_shares 2: SELECT dummy,from_user FROM imap_acl_anyone 3: BEGIN 4: DELETE FROM imap_acl_shares WHERE to_user = 'user1' AND from_user = 'user1' 5: INSERT INTO imap_acl_shares (dummy,to_user,from_user) VALUES ('1','user1','user1') ON DUPLICATE KEY UPDATE dummy='1' 6: DELETE FROM imap_acl_shares WHERE to_user = 'user2' AND from_user = 'user1' 7: INSERT INTO imap_acl_shares (dummy,to_user,from_user) VALUES ('1','user2','user1') ON DUPLICATE KEY UPDATE dummy='1' 8: COMMIT Line 2 is okay, since imap_acl_anyone is always empty (I'm not able to disable this query, otherwise dovecot spawns errors about a missing map, but I can live with this). Don't ask me what lines 4/5 are repeated in 6/7, but these use indexes and I can live with this. But what the hell is line 1? Why doesn't dovecot use a WHERE clause here with either to_user='user1', to_user='user2', from_user='user1', or from_user='user2'? All these WHERE clauses would use an index and speedup the query. Without the WHERE clause I expect 100000 or more results on our production system, which could slow down it massively. Here's my setup: /etc/dovecot/dovecot-dict-sql.conf: connect = host=foo dbname=mail user=imap password=hidden map { pattern = shared/shared-boxes/user/$to/$from table = imap_acl_shares value_field = dummy fields { from_user = $from to_user = $to } } map { pattern = shared/shared-boxes/anyone/$from table = imap_acl_anyone value_field = dummy fields { from_user = $from } } And dovecot -n: # 2.1.6: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_negative_ttl = 5 mins auth_cache_size = 100 M auth_cache_ttl = 5 mins auth_default_realm = example.com auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_gid = 999 mail_location = maildir:~/mail mail_plugins = acl quota mail_uid = 999 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mmap_disable = yes namespace { list = children location = maildir:%%h/mail:INDEX=~/mail/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/passwd.master driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl quota = maildir sieve = ~/.dovecot.sieve sieve_dir = ~/mail/sieve sieve_extensions = +imapflags } protocols = imap pop3 sieve service auth { unix_listener auth-master { group = vmail mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap { process_limit = 4000 } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 100 } service pop3 { process_limit = 1000 } shutdown_clients = no ssl = no syslog_facility = local2 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 10 mail_plugins = quota imap_quota acl imap_acl } protocol pop3 { mail_plugins = quota pop3_no_flag_updates = yes pop3_uidl_format = %v.%u } Tschoeeee Roland From rblayzor.bulk at inoc.net Mon Jul 16 15:58:02 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Mon, 16 Jul 2012 08:58:02 -0400 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS Message-ID: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> In reading the Dovecot NFS WIKI entry: http://wiki2.dovecot.org/NFS It mentions a FreeBSD NFS client caching bug. I believe that bug only exists pre-8.x ? It seems that a patch in the PR notes was MFC'd and made it into 8.x at some point. (at least I checked the source in 8.3 and it was there). So I assume that FreeBSD 8.x mitigates the issue? Maybe not? So our deployment is multi server Dovecot/Exim running on FreeBSD servers, frontended by a layer3 load balancing switch and a NetApp NFSv3 datastore. The only real concern we have right now is race conditions and file write contention on the Dovecot index files. Our original plan was to use Dovecot LMTP on the servers to deliver to the inbox, but that doesn't seem to be recommended as we'll have a number of servers all acting as primary MX for domains and delivering to the same set of mailboxes. So are the Dovecot dotlock file locking mechanisms not enough for multiple servers trying to write/access index files in Maildirs? If we have to deliver straight to the ~/Maildir/new out of Exim, that's fine, also. We're just wondering if there is a added benefit of actually using LMTP. (other than making sure the index are kept the most recent). The other concern we have is client mailbox access. While we can make sessions "sticky" from one IP so they hit the same server, it doesn't help with the person that checks their email via POP from their PC while having their mobile phone access it via IMAP. (obviously from different IP addresses). So I guess there is some possibly index file write contention there as well. (though hopefully this should be rare). I'm curious as to know others experiences with a similar deployment. TIA -- Robert Blayzor INOC, LLC rblayzor at inoc.net http://www.inoc.net/~rblayzor/ From robert at schetterer.org Mon Jul 16 16:15:00 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Jul 2012 15:15:00 +0200 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> Message-ID: <500413D4.8090809@schetterer.org> Am 16.07.2012 14:58, schrieb Robert Blayzor: > In reading the Dovecot NFS WIKI entry: > > http://wiki2.dovecot.org/NFS > > It mentions a FreeBSD NFS client caching bug. I believe that bug only exists pre-8.x ? It seems that a patch in the PR notes was MFC'd and made it into 8.x at some point. (at least I checked the source in 8.3 and it was there). So I assume that FreeBSD 8.x mitigates the issue? Maybe not? > > > So our deployment is multi server Dovecot/Exim running on FreeBSD servers, frontended by a layer3 load balancing switch and a NetApp NFSv3 datastore. > > The only real concern we have right now is race conditions and file write contention on the Dovecot index files. Our original plan was to use Dovecot LMTP on the servers to deliver to the inbox, but that doesn't seem to be recommended as we'll have a number of servers all acting as primary MX for domains and delivering to the same set of mailboxes. So are the Dovecot dotlock file locking mechanisms not enough for multiple servers trying to write/access index files in Maildirs? > > If we have to deliver straight to the ~/Maildir/new out of Exim, that's fine, also. We're just wondering if there is a added benefit of actually using LMTP. (other than making sure the index are kept the most recent). > > The other concern we have is client mailbox access. While we can make sessions "sticky" from one IP so they hit the same server, it doesn't help with the person that checks their email via POP from their PC while having their mobile phone access it via IMAP. (obviously from different IP addresses). So I guess there is some possibly index file write contention there as well. (though hopefully this should be rare). > > I'm curious as to know others experiences with a similar deployment. > > TIA > > There where hot disputs relating such setups, read list archive i guess nfs is simply not the ideal solution, however its used in real existing setups, also maildir may not ideal for good advice there should be more info about how many mailboxes , how much traffic, mailbox size , loadbalancing solution etc dont think exim is critical envolved in this ( but i dont know exim very good ) however someone from the list with equal setup may share his experience with you as far i remember, Timo posted good advice also ,for nfs setups in the past -- Best Regards MfG Robert Schetterer From lazy404 at gmail.com Mon Jul 16 16:40:04 2012 From: lazy404 at gmail.com (Lazy) Date: Mon, 16 Jul 2012 15:40:04 +0200 Subject: [Dovecot] outlook and redirect sieve Message-ID: Hi Aall, we are having issues with outlook clients, running exchange or lotus being unable to parse the email messages redirected by dovecot's sieve 0.3.1, the command is plain redirect "email at lotusmail.xxx"; redirects from mta also work correctly exchange was fixed by removing this commit http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-diff/082216ad12d6/src/lib-sieve/cmd-redirect.c are there any other modifications to the email body or headers that can make those mesages not parsable for outlook ? looking at http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-diff/082216ad12d6/src/lib-sieve/cmd-redirect.c maybe removing HEADER_FILTER_NO_CR fromi_stream_create_header_filter() will make redirected messages more in tact ? what can I do to make redirect more verbatim ? Thanks in advance for any information. Regards, Michal Grzedzicki From rblayzor.bulk at inoc.net Mon Jul 16 17:19:10 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Mon, 16 Jul 2012 10:19:10 -0400 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <500413D4.8090809@schetterer.org> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> <500413D4.8090809@schetterer.org> Message-ID: <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> On Jul 16, 2012, at 9:15 AM, Robert Schetterer wrote: > There where hot disputs relating such setups, read list archive > > i guess nfs is simply not the ideal solution, however its used in real > existing setups, also maildir may not ideal > > for good advice there should be more info about how many mailboxes , how > much traffic, mailbox size , loadbalancing solution etc > > dont think exim is critical envolved in this ( but i dont know exim very > good ) > > however someone from the list with equal setup may share his experience > with you > > as far i remember, Timo posted good advice also ,for nfs setups in the past We're talking about tens of thousands of mailboxes and at least half a dozen servers involved. There is quite a bit of traffic. We've been running Maildir via NFS with another mailserver product for some time, and it's been ok as far as the shared file system standpoint. I don't know why Maildir would be "not ideal". It seems to be the only option when it comes to NFS. NFS is used extensively in our network and we already have a major investment with NetApp, and have been very happy with them. The only issue I see right now is Dovecot index file write bashing if a user accesses their mailbox from multiple locations at the same time, and I gave that example in the previous post. That seems to be more and more common these days but still not the norm for 80%+ of the mailboxes we have. From the MTA side of things, it looks like having Exim write directly into the ~/Maildir/new/ is the only sane choice since mail can be coming into a mailbox from a number of servers at the same time. We've used this extensively in the past with no issues. We use a Brocade ADX to load balancing POP and IMAP sessions to the server. Stickiness doesn't really buy you anything on POP3 since a session connects, does it's thin and disconnects. IMAP on the other hand, I believe can have multiple connections open from the same client, so I think sticky would help. But neither of those help when the client is connecting from a different IP address. Robert Blayzor INOC, LLC rblayzor at inoc.net http://www.inoc.net/~rblayzor/ From lazy404 at gmail.com Mon Jul 16 18:00:13 2012 From: lazy404 at gmail.com (Lazy) Date: Mon, 16 Jul 2012 17:00:13 +0200 Subject: [Dovecot] outlook and redirect sieve In-Reply-To: References: Message-ID: 2012/7/16 Lazy : > Hi Aall, > > we are having issues with outlook clients, running exchange or lotus > being unable to parse the email messages redirected by > > dovecot's sieve 0.3.1, the command is plain redirect "email at lotusmail.xxx"; > redirects from mta also work correctly > > > exchange was fixed by removing this commit > http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-diff/082216ad12d6/src/lib-sieve/cmd-redirect.c > > are there any other modifications to the email body or headers that > can make those mesages not parsable for outlook ? > > looking at http://hg.rename-it.nl/dovecot-2.1-pigeonhole/raw-diff/082216ad12d6/src/lib-sieve/cmd-redirect.c > > maybe removing HEADER_FILTER_NO_CR fromi_stream_create_header_filter() > will make redirected messages more in tact ? > > what can I do to make redirect more verbatim ? > > Thanks in advance for any information. i have got it if ( recipient != NULL ) rfc2822_header_field_write(f, "X-Sieve-Redirected-From", recipient); was adding crlf in the headers, and it made whole email inconsistent (when I have removed crlf filter before) Regards, Michal Grzedzicki From stephan at rename-it.nl Mon Jul 16 18:04:50 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 16 Jul 2012 17:04:50 +0200 Subject: [Dovecot] outlook and redirect sieve In-Reply-To: References: Message-ID: <50042D92.1020700@rename-it.nl> On 7/16/2012 5:00 PM, Lazy wrote: > 2012/7/16 Lazy : > > i have got it > > if ( recipient != NULL ) > rfc2822_header_field_write(f, > "X-Sieve-Redirected-From", recipient); > > was adding crlf in the headers, and it made whole email inconsistent > (when I have removed crlf filter before) Could you explain this a little further? Is this a bug? Regards, Stephan. From rrosenfeld at netcologne.de Mon Jul 16 18:10:36 2012 From: rrosenfeld at netcologne.de (Roland Rosenfeld) Date: Mon, 16 Jul 2012 17:10:36 +0200 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> <500413D4.8090809@schetterer.org> <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> Message-ID: <20120716151036.GA23882@sys-241.netcologne.de> > The only issue I see right now is Dovecot index file write bashing > if a user accesses their mailbox from multiple locations at the same > time, and I gave that example in the previous post. That seems to > be more and more common these days but still not the norm for 80%+ > of the mailboxes we have. We have a similar setup and solved the problem for us this way: - The delivering mail servers (postfix+dovecot) don't write any index files, so we can use round robin here. - We have an additional layer of proxy servers between the loadbalancers and the dovecot IMAP servers, that try to make sessions sticky. For this we use the IMAP/POP3 proxy code of nginx with a PHP script that does some database voodoo, normalizes the user name and based on all this decides which IMAP backend server to use. This is based on the username (not the IP), so it is sticky for multiple IMAP and POP3 connections from different IPs. I think dovecot director should be able to do similar things (our setup is a little bit older and needs some more flexibility). Tschoeeee Roland -- Roland Rosenfeld - Content Delivery - NED - Technik NetCologne Gesellschaft f?r Telekommunikation mbH - HRB 25580, AG K?ln Am Coloneum 9 50829 K?ln Tel.: +49 221 2222-373 Fax: +49 221 2222-7373 Gesch?ftsf?hrer: Dr. Hans Konle (Sprecher), Karl-Heinz Zankel From robert at schetterer.org Mon Jul 16 18:13:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Mon, 16 Jul 2012 17:13:12 +0200 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> <500413D4.8090809@schetterer.org> <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> Message-ID: <50042F88.4010701@schetterer.org> Am 16.07.2012 16:19, schrieb Robert Blayzor: > > We're talking about tens of thousands of mailboxes and at least half a dozen servers involved. There is quite a bit of traffic. We've been running Maildir via NFS with another mailserver product for some time, and it's been ok as far as the shared file system standpoint. jep ,thats a big setup > > I don't know why Maildir would be "not ideal". It seems to be the only option when it comes to NFS. mdbox may be more performant http://wiki2.dovecot.org/MailboxFormat/dbox > > NFS is used extensively in our network and we already have a major investment with NetApp, and have been very happy with them. i believe you, but good cluster file systems may better fit with imap > > The only issue I see right now is Dovecot index file write bashing if a user accesses their mailbox from multiple locations at the same time, and I gave that example in the previous post. That seems to be more and more common these days but still not the norm for 80%+ of the mailboxes we have. in modern times, people use to have many clients open ,same time, over imap i.e with mobiles comming over different ips, your setup solution should honor this, indexes then may get a problem, which you have to handle with http://wiki2.dovecot.org/NFS --snip index files If you keep the index files stored on NFS, you'll need to set mmap_disable=yes. If you're not running lockd you'll have to set lock_method=dotlock, but this degrades performance. Note that some NFS installations have problems with lockd. If you're beginning to get all kinds of locking related errors, try if the problems go away with dotlocking. With mbox/Maildir format it's also possible to store index files on local disk instead of on NFS. If the user gets redirected to different servers, the local indexes are automatically created/updated. If the user is (nearly) always redirected to the same server this should be fine and you would likely get higher performance than indexes stored on NFS, but if the server changes it can be slow to recreate the index/cache files. --snip > > From the MTA side of things, it looks like having Exim write directly into the ~/Maildir/new/ is the only sane choice since mail can be coming into a mailbox from a number of servers at the same time. We've used this extensively in the past with no issues. dont know exim, i use postfix lmtp, and a cluster file system storage drbd, no failures so far > > We use a Brocade ADX to load balancing POP and IMAP sessions to the server. dont know brocade, but good working loadbalancers is a must have , you may also think about using directors additional http://wiki2.dovecot.org/Director shouldnt harm you Stickiness doesn't really buy you anything on POP3 since a session connects, does it's thin and disconnects. IMAP on the other hand, I believe can have multiple connections open from the same client, so I think sticky would help. But neither of those help when the client is connecting from a different IP address. > > > Robert Blayzor > INOC, LLC > rblayzor at inoc.net > http://www.inoc.net/~rblayzor/ > > > i think your setup is quite common, anyway it may not be ideal i would recommand to hire dovecot specialist and/or Timos Company directly to do consulting with such big setups -- Best Regards MfG Robert Schetterer From CMarcus at Media-Brokers.com Mon Jul 16 18:28:54 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 16 Jul 2012 11:28:54 -0400 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> <500413D4.8090809@schetterer.org> <78CAAE70-A3A9-4E7F-888C-BED8640357E0@inoc.net> Message-ID: <50043336.7060309@Media-Brokers.com> On 2012-07-16 10:19 AM, Robert Blayzor wrote: > The only issue I see right now is Dovecot index file write bashing if a user accesses their mailbox from multiple locations at the same time, and I gave that example in the previous post. That seems to be more and more common these days but still not the norm for 80%+ of the mailboxes we have. My understanding is that this (problems with NFS) is precisely what Timo wrote the Director functionality for... http://wiki2.dovecot.org/Director -- Best regards, Charles From warden at geneseo.edu Mon Jul 16 18:34:42 2012 From: warden at geneseo.edu (David Warden) Date: Mon, 16 Jul 2012 11:34:42 -0400 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> Message-ID: <7EF942E5-D954-4579-A392-AC3D4325A8ED@geneseo.edu> On Jul 16, 2012, at 8:58 AM, Robert Blayzor wrote: > In reading the Dovecot NFS WIKI entry: > > http://wiki2.dovecot.org/NFS > > It mentions a FreeBSD NFS client caching bug. I believe that bug only exists pre-8.x ? It seems that a patch in the PR notes was MFC'd and made it into 8.x at some point. (at least I checked the source in 8.3 and it was there). So I assume that FreeBSD 8.x mitigates the issue? Maybe not? > > > So our deployment is multi server Dovecot/Exim running on FreeBSD servers, frontended by a layer3 load balancing switch and a NetApp NFSv3 datastore. Our deployment is similar. Dovecot/Exim on Oracle Linux 6.2 (RHEL derivative), layer3 load balancing (pacemaker/ldirectord managing LVS, "sticky" sessions per client IP) and NetApp NFSv3 datastore (maildir). > > The only real concern we have right now is race conditions and file write contention on the Dovecot index files. Our original plan was to use Dovecot LMTP on the servers to deliver to the inbox, but that doesn't seem to be recommended as we'll have a number of servers all acting as primary MX for domains and delivering to the same set of mailboxes. So are the Dovecot dotlock file locking mechanisms not enough for multiple servers trying to write/access index files in Maildirs? I can't speak to FreeBSD but we have had no problems using the NetApp NFS lock manager (lock_method = fcntl) instead of dotlock. Previously we were using dotlock and any skew in clocks on the Dovecot servers would cause higher CPU usage and tons of extra logging. If it works for you, try to use the NetApp NFS lock manager instead of dotlock. It sounds like you have a bunch of hosts running both Exim and Dovecot and they are all your MX hosts? If so, it sounds like you could use Dovecot's LDA binary or LMTP. Since you are running Dovecot also on your Exim hosts and thus don't need to go over the network to a different host to see the mailstore/deliver the message, I would stick with LDA but both options will work. > > If we have to deliver straight to the ~/Maildir/new out of Exim, that's fine, also. We're just wondering if there is a added benefit of actually using LMTP. (other than making sure the index are kept the most recent). > I believe the latest word from Timo is that the benefit of index updating from LDA/LMTP delivery for maildir is minimal, especially if your indexes are on NFS. We do use LDA for delivery with indexes on NFS, and have not had problems (~7000 user site). The important settings are: # Don't use mmap() at all. This is required if you store indexes to shared # filesystems (NFS or clustered filesystem). #mmap_disable = no mmap_disable = yes # When to use fsync() or fdatasync() calls: # optimized (default): Whenever necessary to avoid losing important data # always: Useful with e.g. NFS when write()s are delayed # never: Never use it (best performance, but crashes can lose data) #mail_fsync = optimized mail_fsync = always # Mail storage exists in NFS. Set this to yes to make Dovecot flush NFS caches # whenever needed. If you're using only a single mail server this isn't needed. #mail_nfs_storage = no mail_nfs_storage = yes # Mail index files also exist in NFS. Setting this to yes requires # mmap_disable=yes and fsync_disable=no. #mail_nfs_index = no mail_nfs_index = yes > The other concern we have is client mailbox access. While we can make sessions "sticky" from one IP so they hit the same server, it doesn't help with the person that checks their email via POP from their PC while having their mobile phone access it via IMAP. (obviously from different IP addresses). So I guess there is some possibly index file write contention there as well. (though hopefully this should be rare). We make sessions "sticky" and have the same "different client IP, same username" pattern. We haven't had problems with this setup since switching away from dotlock to NFS locks. > > I'm curious as to know others experiences with a similar deployment. > I should mention we are hoping to change this deployment soon. As others have said, you would probably be better served at least with Dovecot Director (to ensure client access and Exim deliveries using LMTP for user X always hits server Y) and possibly mdbox instead of maildir. We have been testing Director and I feel good enough about it to move indexes off of NFS onto local disk and setting a scheduled task to update indexes on every server at some reasonable interval so a server failure doesn't require a complete index rebuild when users tied to the failed server get sent to a different one. Migrating from maildir to mdbox is on my to-investigate list once Director is implemented but without having tested a migration there is a lot to like in terms of Alternate Storage and reduced IOPS v Maildir. You just have to *really* trust those index files because they are the only place flags/keywords are stored. > TIA > > -- > Robert Blayzor > INOC, LLC > rblayzor at inoc.net > http://www.inoc.net/~rblayzor/ > > > HTH, David Warden From rblayzor.bulk at inoc.net Mon Jul 16 18:59:06 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Mon, 16 Jul 2012 11:59:06 -0400 Subject: [Dovecot] Multi Server Exim/Dovecot and NFS In-Reply-To: <7EF942E5-D954-4579-A392-AC3D4325A8ED@geneseo.edu> References: <53DDFD25-1908-45DC-82A0-DCEFF7755DD0@inoc.net> <7EF942E5-D954-4579-A392-AC3D4325A8ED@geneseo.edu> Message-ID: On Jul 16, 2012, at 11:34 AM, David Warden wrote: > I can't speak to FreeBSD but we have had no problems using the NetApp NFS lock manager (lock_method = fcntl) instead of dotlock. Previously we were using dotlock and any skew in clocks on the Dovecot servers would cause higher CPU usage and tons of extra logging. If it works for you, try to use the NetApp NFS lock manager instead of dotlock. Right. I believe Pre 7.x on FreeBSD, NFS client side locking was rather dodgy or non-existent. I believe in more recent version 8.x+ (and perhaps even some in 7.x), NFS locks client to server locks are supported. But this is rather encouraging and we'll do some testing with it. I wasn't a fan of going with dotlock... > It sounds like you have a bunch of hosts running both Exim and Dovecot and they are all your MX hosts? If so, it sounds like you could use Dovecot's LDA binary or LMTP. Since you are running Dovecot also on your Exim hosts and thus don't need to go over the network to a different host to see the mailstore/deliver the message, I would stick with LDA but both options will work. Yes, Exim/Dovecot same servers, multiple servers. I had LDA/LMTP working fine until I read the notes that if mail is being delivered simultaneously from multiple servers that this should not be used as it's not safe for the indexing. If fcntl locking fixes this, then we'll look at it again. I was weighing the benefits of doing index updating at STMP time vs just doing it when the user logs in, and I'm leaning toward just having Exim deliver directly to the ~/Maildir/new instead of using LDA. If a user gets 100 emails in their box, but doesn't login for days, no sense in updating all of the index files 100 times at SMTP time, and just do it once when they login. Flawed logic? Also LMTP is a bit more protocol over and is another database lookup on the Dovecot side, etc. > I believe the latest word from Timo is that the benefit of index updating from LDA/LMTP delivery for maildir is minimal, especially if your indexes are on NFS. We do use LDA for delivery with indexes on NFS, and have not had problems (~7000 user site). The important settings are: And that's exactly what I'm thinking. > # Don't use mmap() at all. This is required if you store indexes to shared > # filesystems (NFS or clustered filesystem). > #mmap_disable = no > mmap_disable = yes > > # When to use fsync() or fdatasync() calls: > # optimized (default): Whenever necessary to avoid losing important data > # always: Useful with e.g. NFS when write()s are delayed > # never: Never use it (best performance, but crashes can lose data) > #mail_fsync = optimized > mail_fsync = always > > # Mail storage exists in NFS. Set this to yes to make Dovecot flush NFS caches > # whenever needed. If you're using only a single mail server this isn't needed. > #mail_nfs_storage = no > mail_nfs_storage = yes > # Mail index files also exist in NFS. Setting this to yes requires > # mmap_disable=yes and fsync_disable=no. > #mail_nfs_index = no > mail_nfs_index = yes Righto, we've got all that. Except right now we're just testing with dotlocking... I'm going to have to try fcntl locking and see what that gets me. Perhaps someone here is using FreeBSD with nfs locking now and can share experiences as well. > We make sessions "sticky" and have the same "different client IP, same username" pattern. We haven't had problems with this setup since switching away from dotlock to NFS locks. This is encouraging news. Keeping a client sticky is important I think, especially for IMAP... no so much for POP. I guess in the load balancer you could create a client protocol group and make them all hit the same server, but doing pop/IMAP from the same client would be unusual unless it was several different client types behind NAT. > I should mention we are hoping to change this deployment soon. As others have said, you would probably be better served at least with Dovecot Director (to ensure client access and Exim deliveries using LMTP for user X always hits server Y) and possibly mdbox instead of maildir. We have been testing Director and I feel good enough about it to move indexes off of NFS onto local disk and setting a scheduled task to update indexes on every server at some reasonable interval so a server failure doesn't require a complete index rebuild when users tied to the failed server get sent to a different one. Migrating from maildir to mdbox is on my to-investigate list once Director is implemented but without having tested a migration there is a lot to like in terms of Alternate Storage and reduced IOPS v Maildir. You just have to *really* trust those index files because they are the only place flags/keywords are stored. I'm not really up to speed on the dbox format, but I did look at it briefly. I'd like to stick with maildir if at all possible. It's worked for us for years, even over NFS, with little or no problems. I think your always going to have problems with clients accessing from multiple locations/servers, etc... It's just how complex you want to get. Director seems like a good step, but also adds another layer for us to test and maintain. We're looking at that too.. Thanks for your input! -- Robert Blayzor INOC, LLC rblayzor at inoc.net http://www.inoc.net/~rblayzor/ From lists at wildgooses.com Mon Jul 16 20:45:09 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 16 Jul 2012 18:45:09 +0100 Subject: [Dovecot] bcypt availability In-Reply-To: <1342433109.3843.4.camel@tardis> References: <500289ED.3060705@wildgooses.com> <50030CAC.9000302@r.paypc.com> <1342433109.3843.4.camel@tardis> Message-ID: <50045325.7010507@wildgooses.com> On 16/07/2012 11:05, Noel Butler wrote: > On Sun, 2012-07-15 at 11:32 -0700, Robin wrote: > > >> Indeed. What I have seen is a create deal of variation in the >> configuration (/etc/login.defs or your distro's equivalent) in terms of >> making use of such things. >> >> I don't see any added value to bcrypt over iterated SHA-512, really, and > > > bcrypt and scrypt are password hash's - they are designed to be slow > md5/sha/sha2 are cryptographic hash's - they are designed to be fast But the hash under discussion is sha256crypt, which is a "slow hash" built using sha256 (there is also an sha512crypt) > However, if you keep your database secure, yes, this means using > competent coders, then it matters little what method you use. Yes, but the basis for our discussion is that decent companies with a security budget and reputation to protect have made mistakes, it would be foolish to assume that all our own machines are so much better... The topic is about assuming something goes wrong and a compromise occurs, ie security in depth Today the speeds on single cpus for bcrypt/sha512crypt are in "under 1,000 checks per second" kind of range, so given 4-8 cores per processor you end up with cracking around the "under 10,000 checks per second" kind of range. At present GPUs can test sha512 approx 5x faster than a multicore processor using the latest john the ripper code http://openwall.info/wiki/john/OpenCL-SHA-512 At present bcrypt on GPU is tested at around the same speed as a multicore processor, but a) it's often easier to add multiple GPUs to build a distributed cracker, b) there are estimated performance improvements possible with newer GPUs (bcrypt tries to muddle memory a lot to slow things down, but it doesn't actually do enough to prevent implementation on GPUs). A rough estimate suggested that an upper bound of up to a 10x performance improvement might be possible with the bcrypt on GPU code (probably less, that is a straight instruction for instruction estimate) So at present it seems like sha512crypt is slightly weaker than bcrypt, work will continue on sha256 on GPU in particular (bitcoin...) and can only get faster, possibly this work will benefit sha512 cracking speeds also. However, likely also bcrypt cracking speeds can be improved to within an order of magnitude of sha512 and so they are only a small constant multiple different in performance (change your work factor to make them equivalent...) So my opinion has gone back to being satisfied with sha512crypt. Unfortunately though sha512crypt with default 5,000 rounds is still being broken at rates of 10,000 checks a second on latest GPUs and I personally had a lot of success in the 1990s with dictionaries and breaking original DES crypt at <200 checks a second.... I think if possible it would be desirable to increase the default work factor to something higher than the default, 10,000 checks a second will give up a lot of real user passwords in a reasonable length of time (real users are going to have simple derivatives of dictionary words) Good luck Ed W From lists at wildgooses.com Mon Jul 16 20:49:27 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 16 Jul 2012 18:49:27 +0100 Subject: [Dovecot] Last login datetime on accounts In-Reply-To: <5003F480.4030907@schetterer.org> References: <5003B88C.6020907@schetterer.org> <5003F16C.8090204@Media-Brokers.com> <5003F480.4030907@schetterer.org> Message-ID: <50045427.8070600@wildgooses.com> On 16/07/2012 12:01, Robert Schetterer wrote: > Am 16.07.2012 12:48, schrieb Charles Marcus: >> On 2012-07-16 2:45 AM, Robert Schetterer wrote: >>> i have running touch with 3000 users, i dont see much overhead, anyway >>> its true ,its not very elegant, perhaps i.e you may write some daily >>> cron bash find script looking about latest timestamp of files in .new >>> with maildir >>> which of cource is not the same as last_login script, but may good >>> enough for you >> Why not a simple post-login script that updates your userDB? >> >> http://wiki2.dovecot.org/PostLoginScripting >> > i may fail, but in the orig question > PostLoginScripting is allready used but with touch a last_login file > so update some db with same mech may not much better > i thought some other way then PostLoginScripting was searched for > I have a similar desire, I would also like to log "logout" time. I think for lowest load you would want postloginscript to talk to some long running daemon process, which in turn caches logins and does sensible batching of updates. Timo commented that a plugin would be a good way to do all of the above: http://dovecot.org/patches/2.2/imap-logout-plugin.c I haven't had capacity to build anything yet... Cheers E dW From Jim at themailshack.com Tue Jul 17 00:25:27 2012 From: Jim at themailshack.com (Jim) Date: Mon, 16 Jul 2012 16:25:27 -0500 Subject: [Dovecot] Compile issue Dovecot 2.1.8, OS X 10.6.x Message-ID: Hey all. I'm running Mac OS X 10.6.4, and had Dovecot 2.0.7 installed built from source. I'm now wanting to upgrade to Dovecot 2.1.8, but I can't get MAKE to finish without errors: ./configure --with-mysql (snip) Install prefix . : /usr/local File offsets ... : 64bit I/O polling .... : kqueue I/O notifys .... : kqueue SSL ............ : yes (OpenSSL) GSSAPI ......... : no passdbs ........ : static passwd passwd-file pam checkpassword sql : -shadow -bsdauth -sia -ldap -vpopmail userdbs ........ : static prefetch passwd passwd-file checkpassword sql : -ldap -vpopmail -nss SQL drivers .... : mysql : -pgsql -sqlite Full text search : squat : -lucene -solr sudo make (snip) Undefined symbols: "_libiconv_open", referenced from: _charset_to_utf8_begin in libcharset.a(charset-iconv.o) "_libiconv", referenced from: _charset_to_utf8_reset in libcharset.a(charset-iconv.o) _charset_to_utf8 in libcharset.a(charset-iconv.o) "_libiconv_close", referenced from: _charset_to_utf8_end in libcharset.a(charset-iconv.o) ld: symbol(s) not found collect2: ld returned 1 exit status make[3]: *** [libdovecot.la] Error 1 make[2]: *** [all-recursive] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 The reason for the 'sudo make' was to get around a permissions issue, but I don't think that's related to my issue here. I had build Dovecot 2.0.7 and earlier without issue on this system, but I'm a bit stumped on this particular problem. I also found that the configure parameter --with-sql-drivers no longer seems to work, even though it's still shown in the wiki for 2.x. As far as I can tell, I had used this parameter when compiling 2.0.7. From rblayzor.bulk at inoc.net Tue Jul 17 01:35:06 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Mon, 16 Jul 2012 18:35:06 -0400 Subject: [Dovecot] Selective TLS per local IP Message-ID: <30D71C3F-4356-4B09-A8F3-6E83501B2C27@inoc.net> We have the requirement to provide SSL on some IP addresses, but not others on our servers. Providing SSL is the easy part and we're able to use multiple SSL certificates now. (thanks Timo!) All is working ok, but we several IP hosts that do not require SSL and do not have valid certificates. While we can limit access via a firewall ACL to TLS connect ports (993/995) we can't do so on port 110/143. The problem is that some clients now are smart enough to look for an offered STARTTLS or STLS, and if it's offered, they try to use it. While not normally a problem if your setup for SSL with valid key/certs, if you have a self signed or no CERT at all, it starts connection warnings and errors on the client side. So is there any way possible to turn off advertising of TLS on port or turn it off/on per IP? Something like: ssl = yes ssl_cert = References: Message-ID: <961F8926-34D0-476B-9B92-202340B73DF7@themailshack.com> I just did tried building Dovecot 2.1.0, same issue. Dovecot 2.0.21 was able to build without issues, so this problem seems to be with the entire 2.1.x branch. Jim On Jul 16, 2012, at 4:25 PM, Jim wrote: > Hey all. I'm running Mac OS X 10.6.4, and had Dovecot 2.0.7 installed built from source. I'm now wanting to upgrade to Dovecot 2.1.8, but I can't get MAKE to finish without errors: > > Undefined symbols: > "_libiconv_open", referenced from: > _charset_to_utf8_begin in libcharset.a(charset-iconv.o) > "_libiconv", referenced from: > _charset_to_utf8_reset in libcharset.a(charset-iconv.o) > _charset_to_utf8 in libcharset.a(charset-iconv.o) > "_libiconv_close", referenced from: > _charset_to_utf8_end in libcharset.a(charset-iconv.o) > ld: symbol(s) not found > collect2: ld returned 1 exit status > make[3]: *** [libdovecot.la] Error 1 > make[2]: *** [all-recursive] Error 1 > make[1]: *** [all-recursive] Error 1 > make: *** [all] Error 2 From harri at afaics.de Tue Jul 17 07:26:22 2012 From: harri at afaics.de (Harald Dunkel) Date: Tue, 17 Jul 2012 06:26:22 +0200 Subject: [Dovecot] incremental backups of maildir via rsync In-Reply-To: References: <5003A48F.3050505@afaics.de> Message-ID: <5004E96E.1060609@afaics.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Wojciech, On 07/16/12 08:20, Wojciech Puchar wrote: >> I would like to do incremental backups of the users' maildir directories to a dedicated backup host (using "rsync -SHa" and rsnapshot). > > dont use it. > > use rsync -b option. It is MUCH MUCH BETTER. > > i use it regularly. > I will try. > Split your rsync session if you have huge amount of users or rsync would be taking enormous amount of RAM and have larger chance to fail. > >> Problem is: Dovecot bloats the incremental backup by changing the file names again and again. > > I didn't observe that except when moving from "new" to "cur" > > but --detect-renamed fixes it. > This did not work: # rsync -SHa --detect-renamed /etc/motd /var/tmp rsync: --detect-renamed: unknown option I am using rsync version 3.0.9, but AFAICS there is no such option on the cvs HEAD either. Many thanx Harri -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlAE6WYACgkQUTlbRTxpHjep6gCfQZCOD6t4O1zUkEtKW4j2sPKD OJkAniZAbVWzYhanyeA1koYtyOJU7N53 =X9Jz -----END PGP SIGNATURE----- From harri at afaics.de Tue Jul 17 07:39:54 2012 From: harri at afaics.de (Harald Dunkel) Date: Tue, 17 Jul 2012 06:39:54 +0200 Subject: [Dovecot] incremental backups of maildir via rsync In-Reply-To: <5003B8CC.6040005@schetterer.org> References: <5003A48F.3050505@afaics.de> <5003B8CC.6040005@schetterer.org> Message-ID: <5004EC9A.9000004@afaics.de> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi Robert, On 07/16/12 08:46, Robert Schetterer wrote: > > dsync backup ,may the better the solution in your case > I will surely try. This looks very promising. Many thanx Harri -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.12 (GNU/Linux) iEYEARECAAYFAlAE7JoACgkQUTlbRTxpHjdtzwCfWMX0KGYQTOTzPAJd3VCULGzP e/oAn0Vj0nJE3Z+Qmz1ehxz3cJZ7Zhn3 =/f/4 -----END PGP SIGNATURE----- From kamath at moltingpenguin.com Tue Jul 17 10:32:16 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Tue, 17 Jul 2012 00:32:16 -0700 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <5003F09F.40908@Media-Brokers.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> <5003F09F.40908@Media-Brokers.com> Message-ID: On Jul 16, 2012, at 3:44 AM, Charles Marcus wrote: >>> dovecot -n output maybe > >> Certainly (wasn't on the list of "things to put in a post" on the web >> site, so I missed it): > > Which website? http://www.dovecot.org/mailinglists.html And I missed it, it says "use dovecot -n when showing your configuration". > http://dovecot.org/bugreport.html I was sure it wasn't a bug, it was a screwup on my part in the configuration, so I didn't read the bugreport page. > I have suggested that Timo put a short simple list of troubleshooting suggestions (ie, asking for dovecot -n output and basic non-verbose logs (unless/until asked for verbose logs)) in the welcome message when subscribing to the list (like postfix does)... but in all honesty, most people don't read those either. Well, there could also be a "dovecot-bugreport" script. Much as I hate 'em, sometimes they work... > Regardless, providing configuration details and logs exhibiting the problem is one the most basic requirements when asking for help on any email support list I've ever been on... I really thought it would be a simple "here's what you missed", since the logs are relatively uninformative. . . > Not sure how anyone was supposed to guess that you were using sendmail either... > > And even when you replied to toms message suggesting you provide some logs, you still didn't... > > We'll be happy to try to help you, but we'll need your help in order to do that... Thank you, I recall from years ago how responsive and kind people are on this mailing list. I certainly wasn't trying to be cryptic. Here's the complete set of everything. I wrote a little script to do this repeatedly, to make it easy to make changes and try again. Note that in the last test, it works. Simply adding "X-Original-To: rewards+test at enoent.net" and sending the same message with "/usr/lib/sendmail -t < msg" will do what's expected. Sean ---------------------------------------- Dovecot -n output: ---------------------------------------- Command: dovecot -n ---------------------------------------- # 2.0.13: /etc/dovecot/dovecot.conf # OS: SunOS 5.10 sun4u base_dir = /var/run/dovecot/ hostname = mail.enoent.net lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes lda_original_recipient_header = X-Original-To lmtp_save_to_detail_mailbox = yes mail_location = sdbox:~/Mail:INDEX=/var/dovecot/indices/%u managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date passdb { driver = pam } plugin { recipient_delimiter = + sieve = ~/.dovecot.sieve sieve_dir = ~/.sieve } protocols = imap sieve ssl_cert = , relay=rewards at localhost Jul 17 00:26:53 spector.dmz.enoent.net sendmail[7667]: [ID 702911 mail.info] STARTTLS=server, relay=localhost [127.0.0.1], version=TLSv1/SSLv3, verify=NOT, cipher=DHE-RSA-AES256-SHA, bits=256/256 Jul 17 00:26:53 spector.dmz.enoent.net sendmail[7665]: [ID 702911 mail.info] STARTTLS=client, relay=[127.0.0.1], version=TLSv1/SSLv3, verify=FAIL, cipher=DHE-RSA-AES256-SHA, bits=256/256 Jul 17 00:26:53 spector.dmz.enoent.net milter-greylist: [ID 751742 mail.info] q6H7Qq2g007667: Sender IP 127.0.0.1 and address are SPF-compliant, bypassing greylist Jul 17 00:26:53 spector.dmz.enoent.net sendmail[7667]: [ID 801593 mail.info] q6H7Qq2g007667: from=, size=341, class=0, nrcpts=1, msgid=<201207170726.q6H7Qqld007665 at spector.dmz.enoent.net>, proto=ESMTP, daemon=MTA, relay=localhost [127.0.0.1] Jul 17 00:26:54 spector.dmz.enoent.net amavis[6875]: [ID 702911 mail.notice] (06875) Passed CLEAN {AcceptedInternal}, AM.PDP-SOCK/MYNETS LOCAL [127.0.0.1] [127.0.0.1] -> , Queue-ID: q6H7Qq2g007667, Message-ID: <201207170726.q6H7Qqld007665 at spector.dmz.enoent.net>, mail_id: 6WhbMXFoQ3DP, Hits: 0.248, size: 573, 813 ms Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7667]: [ID 801593 mail.info] q6H7Qq2g007667: Milter insert (0): header: X-Spam-Status: No, score=0.248 tagged_above=-9999 required=6.2\n\ttests=[ALL_TRUSTED=-1, DATE_IN_FUTURE_24_48=1.248] autolearn=no Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7667]: [ID 801593 mail.info] q6H7Qq2g007667: Milter insert (0): header: X-Spam-Level: Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7667]: [ID 801593 mail.info] q6H7Qq2g007667: Milter insert (0): header: X-Spam-Score: 0.248 Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7667]: [ID 801593 mail.info] q6H7Qq2g007667: Milter insert (0): header: X-Spam-Flag: NO Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7667]: [ID 801593 mail.info] q6H7Qq2g007667: Milter add: header: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mail.enoent.net [127.0.0.1]); Tue, 17 Jul 2012 00:26:54 -0700 (PDT) Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7665]: [ID 801593 mail.info] q6H7Qqld007665: to=rewards+test at enoent.net, ctladdr=rewards (1002/1), delay=00:00:02, xdelay=00:00:02, mailer=relay, pri=30089, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (q6H7Qq2g007667 Message accepted for delivery) Jul 17 00:26:54 spector.dmz.enoent.net dovecot: [ID 583609 mail.info] lda(rewards): sieve: msgid=<201207170726.q6H7Qqld007665 at spector.dmz.enoent.net>: stored mail into mailbox 'subtest' Jul 17 00:26:54 spector.dmz.enoent.net sendmail[7670]: [ID 801593 mail.info] q6H7Qq2g007667: to=, ctladdr= (1002/1), delay=00:00:01, xdelay=00:00:00, mailer=local, pri=30933, relay=test, dsn=2.0.0, stat=Sent ======================================== sieve-test(rewards): Info: final result: success ---------------------------------------- Running sieve-test ---------------------------------------- Command: sieve-test /home/rewards/.dovecot.sieve /tmp/email.tst ---------------------------------------- Performed actions: * store message in folder: subtest * store message in folder: plustest Implicit keep: (none) ======================================== ---------------------------------------- Email Message with X-Original-To ---------------------------------------- Command: cat /tmp/email+header ---------------------------------------- X-Original-To: rewards+test at enoent.net From: rewards at enoent.net To: rewards+test at enoent.net Subject: sieve test This is a test ======================================== ---------------------------------------- Syslog Ouput ---------------------------------------- Command: sed 's/< //' /tmp/syslog.diff ---------------------------------------- 4569,4583d4568 Jul 17 00:27:00 spector.dmz.enoent.net sendmail[7686]: [ID 801593 mail.info] q6H7R0eS007686: from=rewards, size=128, class=0, nrcpts=1, msgid=<201207170727.q6H7R0eS007686 at spector.dmz.enoent.net>, relay=rewards at localhost Jul 17 00:27:00 spector.dmz.enoent.net sendmail[7687]: [ID 702911 mail.info] STARTTLS=server, relay=localhost [127.0.0.1], version=TLSv1/SSLv3, verify=NOT, cipher=DHE-RSA-AES256-SHA, bits=256/256 Jul 17 00:27:00 spector.dmz.enoent.net sendmail[7686]: [ID 702911 mail.info] STARTTLS=client, relay=[127.0.0.1], version=TLSv1/SSLv3, verify=FAIL, cipher=DHE-RSA-AES256-SHA, bits=256/256 Jul 17 00:27:00 spector.dmz.enoent.net milter-greylist: [ID 751742 mail.info] q6H7R02o007687: Sender IP 127.0.0.1 and address are SPF-compliant, bypassing greylist Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7687]: [ID 801593 mail.info] q6H7R02o007687: from=, size=380, class=0, nrcpts=1, msgid=<201207170727.q6H7R0eS007686 at spector.dmz.enoent.net>, proto=ESMTP, daemon=MTA, relay=localhost [127.0.0.1] Jul 17 00:27:01 spector.dmz.enoent.net amavis[6862]: [ID 702911 mail.notice] (06862) Passed CLEAN {AcceptedInternal}, AM.PDP-SOCK/MYNETS LOCAL [127.0.0.1] [127.0.0.1] -> , Queue-ID: q6H7R02o007687, Message-ID: <201207170727.q6H7R0eS007686 at spector.dmz.enoent.net>, mail_id: NQO5yujz1eQp, Hits: 0.248, size: 613, 794 ms Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7687]: [ID 801593 mail.info] q6H7R02o007687: Milter insert (0): header: X-Spam-Status: No, score=0.248 tagged_above=-9999 required=6.2\n\ttests=[ALL_TRUSTED=-1, DATE_IN_FUTURE_24_48=1.248] autolearn=no Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7687]: [ID 801593 mail.info] q6H7R02o007687: Milter insert (0): header: X-Spam-Level: Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7687]: [ID 801593 mail.info] q6H7R02o007687: Milter insert (0): header: X-Spam-Score: 0.248 Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7687]: [ID 801593 mail.info] q6H7R02o007687: Milter insert (0): header: X-Spam-Flag: NO Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7687]: [ID 801593 mail.info] q6H7R02o007687: Milter add: header: X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (mail.enoent.net [127.0.0.1]); Tue, 17 Jul 2012 00:27:01 -0700 (PDT) Jul 17 00:27:01 spector.dmz.enoent.net sendmail[7686]: [ID 801593 mail.info] q6H7R0eS007686: to=rewards+test at enoent.net, ctladdr=rewards (1002/1), delay=00:00:01, xdelay=00:00:01, mailer=relay, pri=30128, relay=[127.0.0.1] [127.0.0.1], dsn=2.0.0, stat=Sent (q6H7R02o007687 Message accepted for delivery) Jul 17 00:27:02 spector.dmz.enoent.net dovecot: [ID 583609 mail.info] lda(rewards): sieve: msgid=<201207170727.q6H7R0eS007686 at spector.dmz.enoent.net>: stored mail into mailbox 'subtest' Jul 17 00:27:02 spector.dmz.enoent.net dovecot: [ID 583609 mail.info] lda(rewards): sieve: msgid=<201207170727.q6H7R0eS007686 at spector.dmz.enoent.net>: stored mail into mailbox 'plustest' Jul 17 00:27:02 spector.dmz.enoent.net sendmail[7689]: [ID 801593 mail.info] q6H7R02o007687: to=, ctladdr= (1002/1), delay=00:00:02, xdelay=00:00:01, mailer=local, pri=30971, relay=test, dsn=2.0.0, stat=Sent ======================================== From wojtek at wojtek.tensor.gdynia.pl Tue Jul 17 12:32:09 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Tue, 17 Jul 2012 11:32:09 +0200 (CEST) Subject: [Dovecot] incremental backups of maildir via rsync In-Reply-To: <5004E96E.1060609@afaics.de> References: <5003A48F.3050505@afaics.de> <5004E96E.1060609@afaics.de> Message-ID: >> but --detect-renamed fixes it. >> > > This did not work: > > # rsync -SHa --detect-renamed /etc/motd /var/tmp > rsync: --detect-renamed: unknown option rsync 3.0.9 FreeBSD ports with option "enable detect renamed" it uses rsync-patches-3.0.9.tar.gz in addition to main source. patch for renaming files detection (and many others) are there. it is available from rsync site http://rsync.samba.org/ftp/rsync/rsync-patches-3.0.9.tar.gz From CMarcus at Media-Brokers.com Tue Jul 17 13:18:50 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 17 Jul 2012 06:18:50 -0400 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> <5003F09F.40908@Media-Brokers.com> Message-ID: <50053C0A.5090208@Media-Brokers.com> On 2012-07-17 3:32 AM, Sean Kamath wrote: > I was sure it wasn't a bug, it was a screwup on my part in the > configuration, so I didn't read the bugreport page. Actually, although it is called 'bugreport', you get to it via the 'Troubleshooting' link on the wiki... But no worries, you've now provided everything that should be needed (I know nothing about sendmail, and won't be much help for the rest, but someone else will likely be able to see what is happening)... > I really thought it would be a simple "here's what you missed", since > the logs are relatively uninformative... The point is, what may appear to be uninformative to you may tell Timo (or someone else who knows what to look for) all they need to know to pinpoint the problem. >I certainly wasn't trying to be cryptic. I know, but as you can see, the dovecot -n output really tells a lot - like what version you're on, and even what OS and filesystem. > Here's the complete set of everything. I wrote a little script to do > this repeatedly, to make it easy to make changes and try again. One thing I did notice is that you are sending this mail from an internal IP... since most people use very different checks for such mail (bypassing many/most checks), maybe you should be testing from an external address (one that comes in from outside your network)? > Note that in the last test, it works. Simply adding "X-Original-To: > rewards+test at enoent.net" and sending the same message with > "/usr/lib/sendmail -t< msg" will do what's expected. > Command: dovecot -n > ---------------------------------------- > # 2.0.13: /etc/dovecot/dovecot.conf Also, it is always recommended when setting up a new server to start with the latest version... for 2.0.x that would be 2.0.19, but the latest is 2.1.8... > lda_original_recipient_header = X-Original-To > lmtp_save_to_detail_mailbox = yes Ok, I think this is the problem... Dovecot's LMTP currently does NOT preserve the x-original-to header... in fact, brought this up recently (I was one of the ones interested in Timo adding support for it) on this (and the postfix, which is what I use) on this very list: http://www.dovecot.org/list/dovecot/2012-April/065075.html Apparently, after Timo discussed this on the postfix list, he determined that it would not be a very significant change for him to make to add support for this header in LMTP and that he would get to it sooner than later, but as of now, it has not been added. -- Best regards, Charles From CMarcus at Media-Brokers.com Tue Jul 17 13:24:42 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 17 Jul 2012 06:24:42 -0400 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <50053C0A.5090208@Media-Brokers.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> <5003F09F.40908@Media-Brokers.com> <50053C0A.5090208@Media-Brokers.com> Message-ID: <50053D6A.6050000@Media-Brokers.com> On 2012-07-17 6:18 AM, Charles Marcus wrote: >> lda_original_recipient_header = X-Original-To >> lmtp_save_to_detail_mailbox = yes > Ok, I think this is the problem... > > Dovecot's LMTP currently does NOT preserve the x-original-to header... > in fact, brought this up recently (I was one of the ones interested in > Timo adding support for it) on this (and the postfix, which is what I > use) on this very list: > > http://www.dovecot.org/list/dovecot/2012-April/065075.html > > Apparently, after Timo discussed this on the postfix list, he determined > that it would not be a very significant change for him to make to add > support for this header in LMTP and that he would get to it sooner than > later, but as of now, it has not been added. Ok, that's what I get for replying to an email like this before having at least my first cup of coffee... ;) What Timo said was that LMTP doesn't currently have support for *adding* the headers, but it doesn't remove them either, so I misspoke when I said it doesn't *preserve* the headers - it does. I had asked the question specifically because postfix's LMTP support does *not* provide the header, so Timo figured a way to add support for adding it in dovecot's LMTP implementation (but it currently doesn't have it). So, since apparently sendmail *does* add it to its LMTP implementation, and apparently your sieve script works properly when it is present, you need it for it to work correctly. Why, I do not know, but someone else with more knowledge can probably answer that one now. -- Best regards, Charles From joseba.torre at ehu.es Tue Jul 17 13:43:05 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 17 Jul 2012 12:43:05 +0200 Subject: [Dovecot] doveadm director status username != doveadm director status username@mailserver Message-ID: <500541B9.30402@ehu.es> Hi, I've almost finished my new director based setup, but in the first test I discovered that imap and lmtp connections were not always being proxied to the same server. After some research now I think that the main problem is that in imap connections users connect as 'username' while lmtp connections are as 'username at mailserver'. In my current setup I receive mail via SMTP in my director servers (which are loadbalanced by a CISCO machine), and then they send them to dovecot running on localhost. I've tried to get rid of the @mailserver in postfix in a generic way with no success. Is there any way to redirect connections based only in the local part of the address? Or there is a way to get rid of the @mailserver in the postfix -> lmtp connection? Both doveadm config -n and postconf -n are attached. Thanks. $ doveadm config -n # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.17.1.el6.x86_64 x86_64 Red Hat Enterprise Linux Server release 6.2 (Santiago) auth_debug = yes auth_master_user_separator = * auth_verbose = yes director_mail_servers = buzones2.ehu.es buzones3.ehu.es director_servers = director1.ehu.es director2.ehu.es doveadm_proxy_port = 4444 lmtp_proxy = yes mail_debug = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = proxy=y nopassword=y starttls=any-cert driver = static } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 3333 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { inet_listener { port = 4444 } } service imap-login { executable = imap-login director } service lmtp { inet_listener ltmp { address = director2.ehu.es 127.0.0.1 port = 24 } unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { executable = managesieve-login director vsz_limit = 128 M } service pop3-login { executable = pop3-login director } ssl_cert = References: <500541B9.30402@ehu.es> Message-ID: <500543C9.7090507@Media-Brokers.com> On 2012-07-17 6:43 AM, Joseba Torre wrote: > Is there any way to redirect connections based only in the local part of > the address? Or there is a way to get rid of the @mailserver in the > postfix -> lmtp connection? First, my imap connections/logins show the full username (user at example.com) in the logs. I'm not using LMTP (yet), but don't think that affects imap logins... Second, since, in a virtual setup, it is very likely that you will have a collision of the local part, shouldn't you be figuring out how to make imap logins use the full user at example.com instead? -- Best regards, Charles From tss at iki.fi Tue Jul 17 13:54:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 13:54:29 +0300 Subject: [Dovecot] doveadm director status username != doveadm director status username@mailserver In-Reply-To: <500541B9.30402@ehu.es> References: <500541B9.30402@ehu.es> Message-ID: <86DDC6D4-6A90-42F7-AF25-104682C865AB@iki.fi> On 17.7.2012, at 13.43, Joseba Torre wrote: > I've almost finished my new director based setup, but in the first test I discovered that imap and lmtp connections were not always being proxied to the same server. After some research now I think that the main problem is that in imap connections users connect as 'username' while lmtp connections are as 'username at mailserver'. > > In my current setup I receive mail via SMTP in my director servers (which are loadbalanced by a CISCO machine), and then they send them to dovecot running on localhost. I've tried to get rid of the @mailserver in postfix in a generic way with no success. > > Is there any way to redirect connections based only in the local part of the address? Or there is a way to get rid of the @mailserver in the postfix -> lmtp connection? director_username_hash = %n From tss at iki.fi Tue Jul 17 13:57:34 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 13:57:34 +0300 Subject: [Dovecot] bcypt availability In-Reply-To: References: Message-ID: On 12.7.2012, at 13.32, Nick Edwards wrote: > Do you intend to introduce bcrypt into the built in password schemes? No, but I would accept a patch for that. From tss at iki.fi Tue Jul 17 13:59:48 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 13:59:48 +0300 Subject: [Dovecot] Selective TLS per local IP In-Reply-To: <30D71C3F-4356-4B09-A8F3-6E83501B2C27@inoc.net> References: <30D71C3F-4356-4B09-A8F3-6E83501B2C27@inoc.net> Message-ID: On 17.7.2012, at 1.35, Robert Blayzor wrote: > So is there any way possible to turn off advertising of TLS on port or turn it off/on per IP? > > Something like: If those work, then yes. If they don't, then no. I'd think they would work. > ssl = yes > ssl_cert = ssl_key = > local 10.1.1.1 { > protocol imap { > ssl_cert = ssl_key = } > protocol pop3 { > ssl_cert = ssl_key = } > } > > local 10.1.1.2 { > ssl = no > } > > > > *or* > > ssl = no > > local 10.1.1.1 { > ssl = yes > protocol imap { > ssl_cert = ssl_key = } > protocol pop3 { > ssl_cert = ssl_key = } > } > > > -- > Robert Blayzor > INOC, LLC > rblayzor at inoc.net > http://www.inoc.net/~rblayzor/ > > > From tss at iki.fi Tue Jul 17 14:06:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:06:56 +0300 Subject: [Dovecot] imapc: Restrict the number of connections In-Reply-To: <500016F3.8060805@necoro.eu> References: <500016F3.8060805@necoro.eu> Message-ID: <138877BE-69CF-4B11-B699-2BAA1F02F7A6@iki.fi> On 13.7.2012, at 15.39, Ren? Neumann wrote: > One problem remains: Though there is only one account that is allowed to > use this imapc-connection, each imap-process opens a connection to the > imapc'd server. > > Is this caused directly by the imapc_*-definitions in the config? Or is > there a way of restricting this to only the account which actually uses it. Make the namespace available only for this one user. How exactly to do this could be a bit tricky/kludgy. See the bottom of http://wiki2.dovecot.org/Namespaces or http://wiki2.dovecot.org/PostLoginScripting From tss at iki.fi Tue Jul 17 14:28:10 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:28:10 +0300 Subject: [Dovecot] SQLite dovecot query caching In-Reply-To: <4FFFB262.7040409@optusnet.com.au> References: <4FE5FEB4.2050801@optusnet.com.au> <1340473147.5967.89.camel@hurina> <4FE608E8.6090106@optusnet.com.au> <1340477828.5967.91.camel@hurina> <4FFFB262.7040409@optusnet.com.au> Message-ID: <9D7331EF-9E49-4ABD-A941-B0874BDA012B@iki.fi> On 13.7.2012, at 8.30, Dominic Malolepszy wrote: >> It caches the passdb lookup. The cache key consists of the given % >> variables in the SQL query. So if your SQL query doesn't contain %n/%u >> then the cache doesn't add per-user entries. > > I had a chance to play around with this in the lab. The cache hit appears to be returning values that should actually be ignored particularly the username for the previous cached entry. This is causing it to rewrite the username using what is in the cache. The simplest solution would be to just not put username to cache, but I'm worried it might break something else.. So for now I don't think I'm going to change it. From tss at iki.fi Tue Jul 17 14:29:31 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:29:31 +0300 Subject: [Dovecot] Difference between proxy_refresh and proxy_timeout in director In-Reply-To: <4FFFE1DA.7020706@um.es> References: <4FFFE1DA.7020706@um.es> Message-ID: <065FFCBD-0C09-4697-991A-A5BB9FE71F5C@iki.fi> On 13.7.2012, at 11.52, Angel L. Mateo wrote: > I know proxy_timeout is the timeout for the director operation. But what is the purpose for proxy_refresh? Director returns proxy_refresh to login process. The login process then tells director this often that the user's connection still exists. So it's an internal thing that you shouldn't need to care about. From tss at iki.fi Tue Jul 17 14:32:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:32:42 +0300 Subject: [Dovecot] Override userdb=passwd gid In-Reply-To: <4FFED468.5000301@ekp.ee> References: <4FFED468.5000301@ekp.ee> Message-ID: <22D8A40C-780A-4B0A-899F-76DDD7D7873A@iki.fi> On 12.7.2012, at 16.43, Veiko Kukk wrote: > I have problem with Dovecot 2.1.1 writing maildir files as users primary group. Tried to override gid value, but probably i'm missing something... > > userdb { > args = blocking=no gid=mail > driver = passwd > } > > Also tried using 'override_fields = gid=mail'. Users are from nss and winbind. Should work I think. > Any ideas what i might do wrong? Set auth_debug=yes and mail_debug=yes and show the logs. From tss at iki.fi Tue Jul 17 14:38:59 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:38:59 +0300 Subject: [Dovecot] Seq-range order in sequence-set are matter? In-Reply-To: <40f28e5c42b.4117456b@smtp.gmail.com> References: <40f28e5c42b.4117456b@smtp.gmail.com> Message-ID: I can reproduce this in v1.0 and v1.1, but not with v1.2 or newer. So this has been fixed 3 years ago. On 12.7.2012, at 14.13, Lay Andr?s wrote: > Hi! > > I have a mailbox with 12 messages. Theese two commands: > > x fetch 5,8:12 (UID) > * 5 FETCH (UID 10) > * 8 FETCH (UID 13) > * 9 FETCH (UID 14) > * 10 FETCH (UID 15) > * 11 FETCH (UID 16) > * 12 FETCH (UID 17) > x OK Fetch completed. > > and > > x fetch 8:12,5 (UID) > * 5 FETCH (UID 10) > * 8 FETCH (UID 13) > * 9 FETCH (UID 14) > * 10 FETCH (UID 15) > * 11 FETCH (UID 16) > * 12 FETCH (UID 17) > x OK Fetch completed. > > outputs are equeal, this clean. But: > > x fetch 1:*,5 (UID) > * 1 FETCH (UID 4) > * 2 FETCH (UID 5) > * 3 FETCH (UID 6) > * 4 FETCH (UID 7) > * 5 FETCH (UID 10) > * 6 FETCH (UID 11) > * 7 FETCH (UID 12) > * 8 FETCH (UID 13) > * 9 FETCH (UID 14) > * 10 FETCH (UID 15) > * 11 FETCH (UID 16) > * 12 FETCH (UID 17) > x OK Fetch completed. > > and > > x fetch 5,1:* (UID) > * 5 FETCH (UID 10) > * 6 FETCH (UID 11) > * 7 FETCH (UID 12) > * 8 FETCH (UID 13) > * 9 FETCH (UID 14) > * 10 FETCH (UID 15) > * 11 FETCH (UID 16) > * 12 FETCH (UID 17) > x OK Fetch completed. > > command outputs are NOT equal? Why? > > Thank you! > > Lay > From tss at iki.fi Tue Jul 17 14:40:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:40:02 +0300 Subject: [Dovecot] different mail_max_userip settings In-Reply-To: References: Message-ID: <0115B28E-0CC6-483A-93C3-4E9D5774688E@iki.fi> On 10.7.2012, at 10.24, Federico Bianchi wrote: > Is it possible to have mail_max_userip set to a value for localhost (webmail) and to another value for everything else? mail_max_userip_connections = 10 remote 127.0.0.1 { mail_max_userip_connections = 0 } From joseba.torre at ehu.es Tue Jul 17 14:57:46 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 17 Jul 2012 13:57:46 +0200 Subject: [Dovecot] doveadm director status username != doveadm director status username@mailserver In-Reply-To: References: <500541B9.30402@ehu.es> Message-ID: <5005533A.8030707@ehu.es> El 17/07/12 13:34, Lee Standen escribi?: > One thing that stands out to me is the lack of passdb. One of the > recommendations I got from the passdb documentation was to override the > user field, so the user is always logged in consistently. Without this, > "BOB", "bob" and "bob at mydomain.com " might all > accept auth, but be handled differently! If you configure your passdb, > then you'll get some consistency, which will probably fix your director > problem. We don't use passdb in the directors because incoming mails have already been checked by postfix, and the final mail server need to check our ldap server for other values, so we check there. Actually, I still feel that I can override some ldap connections if I think about it carefully. Thank you. > > At the very least, you probably want to lowercase all your usernames. > From tss at iki.fi Tue Jul 17 14:59:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 14:59:35 +0300 Subject: [Dovecot] dovecot 2.1.8 still tries to touch any mountpoint In-Reply-To: <4FFC0B38.9070603@thelounge.net> References: <4FFC0B38.9070603@thelounge.net> Message-ID: <8ABDE463-A651-4692-8946-252C0B5B6864@iki.fi> On 10.7.2012, at 14.00, Reindl Harald wrote: > why does dovecot touch any ever used fuse-mountpoint > and scream at the next start that is no longer mounted? > > "If this is intentional, remove it with doveadm mount" > why if i never called "doveadm" on any setup at all? > > Jul 10 09:10:56 rh dovecot: master: Dovecot v2.1.8 starting up (core dumps disabled) > Jul 10 09:10:56 rh dovecot: master: Warning: /sys/fs/cgroup/cpu is no longer mounted. If this is intentional, > remove it with doveadm mount v2.1.9+ log message will point to http://wiki2.dovecot.org/Mountpoints From joseba.torre at ehu.es Tue Jul 17 15:01:39 2012 From: joseba.torre at ehu.es (Joseba Torre) Date: Tue, 17 Jul 2012 14:01:39 +0200 Subject: [Dovecot] doveadm director status username != doveadm director status username@mailserver In-Reply-To: <500543C9.7090507@Media-Brokers.com> References: <500541B9.30402@ehu.es> <500543C9.7090507@Media-Brokers.com> Message-ID: <50055423.3010802@ehu.es> El 17/07/12 12:51, Charles Marcus escribi?: >> > > First, my imap connections/logins show the full username > (user at example.com) in the logs. I'm not using LMTP (yet), but don't > think that affects imap logins... > > Second, since, in a virtual setup, it is very likely that you will have > a collision of the local part, shouldn't you be figuring out how to make > imap logins use the full user at example.com instead? We are only serving one domain here, so we have no need for the @domain part in the login, so we are not using it. In the short future we will allow people to connect using their full address, but not for now. Timo's solution was exactly was I was looking for. Thank you. From tss at iki.fi Tue Jul 17 15:06:53 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:06:53 +0300 Subject: [Dovecot] doveadm director status username != doveadm director status username@mailserver In-Reply-To: <86DDC6D4-6A90-42F7-AF25-104682C865AB@iki.fi> References: <500541B9.30402@ehu.es> <86DDC6D4-6A90-42F7-AF25-104682C865AB@iki.fi> Message-ID: <7FE26CF8-4F5A-4CF5-A293-691130C234F7@iki.fi> On 17.7.2012, at 13.54, Timo Sirainen wrote: > On 17.7.2012, at 13.43, Joseba Torre wrote: > >> I've almost finished my new director based setup, but in the first test I discovered that imap and lmtp connections were not always being proxied to the same server. After some research now I think that the main problem is that in imap connections users connect as 'username' while lmtp connections are as 'username at mailserver'. >> >> In my current setup I receive mail via SMTP in my director servers (which are loadbalanced by a CISCO machine), and then they send them to dovecot running on localhost. I've tried to get rid of the @mailserver in postfix in a generic way with no success. >> >> Is there any way to redirect connections based only in the local part of the address? Or there is a way to get rid of the @mailserver in the postfix -> lmtp connection? > > director_username_hash = %n BTW. This feature was implemented so that %d value could be used to allow shared mailboxes between domains to work in setups that have many smallish domains. And now that I think about it, the default should probably be %Lu From h.reindl at thelounge.net Tue Jul 17 15:12:06 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Tue, 17 Jul 2012 14:12:06 +0200 Subject: [Dovecot] dovecot 2.1.8 still tries to touch any mountpoint In-Reply-To: <8ABDE463-A651-4692-8946-252C0B5B6864@iki.fi> References: <4FFC0B38.9070603@thelounge.net> <8ABDE463-A651-4692-8946-252C0B5B6864@iki.fi> Message-ID: <50055696.7040907@thelounge.net> Am 17.07.2012 13:59, schrieb Timo Sirainen: > On 10.7.2012, at 14.00, Reindl Harald wrote: > >> why does dovecot touch any ever used fuse-mountpoint >> and scream at the next start that is no longer mounted? >> >> "If this is intentional, remove it with doveadm mount" >> why if i never called "doveadm" on any setup at all? >> >> Jul 10 09:10:56 rh dovecot: master: Dovecot v2.1.8 starting up (core dumps disabled) >> Jul 10 09:10:56 rh dovecot: master: Warning: /sys/fs/cgroup/cpu is no longer mounted. If this is intentional, >> remove it with doveadm mount > > v2.1.9+ log message will point to http://wiki2.dovecot.org/Mountpoints Dovecot already internally filters out many mountpoints and filesystems that are pretty much guaranteed not to contain any emails i bet /sys/fs/cgroup will never contain any mail :-) the bug is taht it also adds fuse-mountpoints where the dovecot suer never ever has any access permissions and whines later if they are unmounted i see no reason for such magic why does a mailserver touch any mountpoint where it is not explicit configured to store mails? _____________________________ Jul 17 14:08:48 rh dovecot: master: Warning: /sys/fs/cgroup/cpu is no longer mounted. If this is intentional, remove it with doveadm mount Jul 17 14:08:48 rh dovecot: master: Warning: /sys/fs/cgroup/cpuacct is no longer mounted. If this is intentional, remove it with doveadm mount Jul 17 14:10:17 srv-rhsoft dovecot: master: Warning: /var/run/vmblock-fuse is no longer mounted. If this is intentional, remove it with doveadm mount -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From rblayzor.bulk at inoc.net Tue Jul 17 15:20:19 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 17 Jul 2012 08:20:19 -0400 Subject: [Dovecot] Selective TLS per local IP In-Reply-To: References: <30D71C3F-4356-4B09-A8F3-6E83501B2C27@inoc.net> Message-ID: On Jul 17, 2012, at 6:59 AM, Timo Sirainen wrote: > On 17.7.2012, at 1.35, Robert Blayzor wrote: > >> So is there any way possible to turn off advertising of TLS on port or turn it off/on per IP? >> >> Something like: > > If those work, then yes. If they don't, then no. I'd think they would work. No, they do not. It would be nice if it did. Or at least some way to disable TLS offering/advertisement if disabling the SSL socket is not possible. -- Robert Blayzor INOC, LLC rblayzor at inoc.net http://www.inoc.net/~rblayzor/ From tss at iki.fi Tue Jul 17 15:21:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:21:01 +0300 Subject: [Dovecot] dovecot 2.1.8 still tries to touch any mountpoint In-Reply-To: <50055696.7040907@thelounge.net> References: <4FFC0B38.9070603@thelounge.net> <8ABDE463-A651-4692-8946-252C0B5B6864@iki.fi> <50055696.7040907@thelounge.net> Message-ID: <7652D4ED-E9AF-42F8-97F5-6DDF1D726171@iki.fi> On 17.7.2012, at 15.12, Reindl Harald wrote: >> v2.1.9+ log message will point to http://wiki2.dovecot.org/Mountpoints > > Dovecot already internally filters out many mountpoints and filesystems that > are pretty much guaranteed not to contain any emails > > i bet /sys/fs/cgroup will never contain any mail :-) v2.1.7+ won't add any mounts from /sys by default. But you had already run an older Dovecot version that had added those mountpoints, so v2.1.8 will still complain about them until you remove them. But they won't get added again even if you don't add an ignore for them. > the bug is taht it also adds fuse-mountpoints where the dovecot > suer never ever has any access permissions and whines later > if they are unmounted It's not easy for Dovecot to know this. Other installations are using FUSE mountpoints to store mails. > i see no reason for such magic > > why does a mailserver touch any mountpoint where it is not > explicit configured to store mails? Because Dovecot doesn't know where mails can be stored (userdb lookups can override the defaults). The other possibility would have been to require you to explicitly list all the mountpoints that can contain mails/indexes and have Dovecot refuse to access the mails outside that list. But that would have broken all the existing installations. > Jul 17 14:10:17 srv-rhsoft dovecot: master: Warning: /var/run/vmblock-fuse is no longer mounted. If this is > intentional, remove it with doveadm mount I guess /var/run and /run could be added to the default ignore list. From tss at iki.fi Tue Jul 17 15:28:56 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:28:56 +0300 Subject: [Dovecot] Selective TLS per local IP In-Reply-To: References: <30D71C3F-4356-4B09-A8F3-6E83501B2C27@inoc.net> Message-ID: <41303680-06C4-41FB-AA9E-40C2945FEA13@iki.fi> On 17.7.2012, at 15.20, Robert Blayzor wrote: > On Jul 17, 2012, at 6:59 AM, Timo Sirainen wrote: >> On 17.7.2012, at 1.35, Robert Blayzor wrote: >> >>> So is there any way possible to turn off advertising of TLS on port or turn it off/on per IP? >>> >>> Something like: >> >> If those work, then yes. If they don't, then no. I'd think they would work. > > > No, they do not. It would be nice if it did. Or at least some way to disable TLS offering/advertisement if disabling the SSL socket is not possible. Here: http://hg.dovecot.org/dovecot-2.1/rev/258c2e231357 From tss at iki.fi Tue Jul 17 15:31:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:31:29 +0300 Subject: [Dovecot] files missed from install In-Reply-To: <4FF46CB3.9050901@tree-of-life.co.uk> References: <4FF46CB3.9050901@tree-of-life.co.uk> Message-ID: On 4.7.2012, at 19.17, Tim Dickson wrote: > version: using dovecot 2.1.8 or 2.0.12 > > bug: installation - some files missed > > details: dovecot-openssl.cnf and mkcert.sh are not copied from the src/doc folder to the installation doc folder on installation (make install) > both are referred to in the documentation and are useful. > > is there any chance of updating the build scripts to include these files? > thanks, I suppose they could be. http://hg.dovecot.org/dovecot-2.1/rev/d01a06d821cf From tss at iki.fi Tue Jul 17 15:33:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:33:36 +0300 Subject: [Dovecot] LDAP.conf filter issue In-Reply-To: References: Message-ID: <6FC89DED-C2FC-4738-8A22-F7029A3FB3EE@iki.fi> On 5.7.2012, at 16.45, Tony Hlabse wrote: > We are trying to use the user_filter options to limit authentication to a single user group. The cusotmer has security rules that they want to only have users belonging to a group to have access. Sample we have tried. > > auth_bind = yes > auth_bind_userdn = EXAMPLE\%n user_filter is used only for userdb lookup, not for authentication. You could use pass_filter to limit authentication, but with the auth_bind_userdn setting enabled that doesn't work because its whole purpose is to skip the pass_filter. From tss at iki.fi Tue Jul 17 15:52:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:52:55 +0300 Subject: [Dovecot] Feature request In-Reply-To: References: Message-ID: On 7.7.2012, at 17.26, Malloc Kilobyte wrote: > Appreciating all Dovecot rich features, I lack just one. And this is the > ability to customize the "quota exceeded, message rejected" message. I know > I can set it's default content using quota_exceeded_message parameter, but > i would also like to have ability to set: > > - to whom the message is sent. Some mail systems send copy not only to the > sender, but also to the recipient. It makes sense in case rejected message > has big size. I've thought about changing the default over-quota handling so that the user would be allowed one final mail to go over quota, as long as the mail is smaller than the user's total quota. And in normal situations user's quota is higher than the max. allowed message size advertised by SMTP server. Then this setting wouldn't be useful. > - it's subject rejection_subject setting can already change this, although it applies also for Sieve rejects. Also in a preferred setup the rejection is done by replying failure to SMTP's RCPT TO command, in which case nothing else can be configured besides the one rejection string. > - if the rejected message is being attached ( with or without it's own > attachments ) or not Spam is often sent this way. > Moreover, I think it would make sense if the "rejected messege" contained > information about size of rejected message and current mailbox space usage. Message size could possibly be added, but I think current space usage could be considered a privacy leak. From tss at iki.fi Tue Jul 17 15:59:18 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 15:59:18 +0300 Subject: [Dovecot] MySQL dict issues In-Reply-To: <4FFDC1A0.6040603@airstreamcomm.net> References: <4FFDC1A0.6040603@airstreamcomm.net> Message-ID: On 11.7.2012, at 21.10, list at airstreamcomm.net wrote: > Testing using the mysql dict for quota storage (dovecot-2.0.12-2_127.el5), and ran into a couple issues. First is a permissions issue: > > dovecot: lmtp(26786, dude at test.tld): Error: net_connect_unix(/var/run/dovecot/dict) failed: Permission denied (euid=5000() egid=5000() missing +r perm: /var/run/dovecot/dict, euid is not dir owner) > > The file /var/run/dovecot/dict is listed with srw------- perms. I changes the perms to srw----rwx and it works, but I would like to figure out how to configure properly to run with the uid and gid 5000 as this is the user for virtual mail. http://wiki2.dovecot.org/Dict explains this. > The second issue is that two identical rows are created in the mysql database for each user quota. The next message delivered updates both rows. Identical? Or perhaps one is the number of bytes and the other is the number of messages? > The last issue might be a documentation omission. Dovecot warned me that the mysql user did not have DELETE privileges on the quota table: > > dovecot: dict: Error: sql dict: commit failed: DELETE command denied to user 'user'@'a.b.c.d' for table 'domain_user_quotas' This is done when recalculating quota. > The quota dict documentation (at http://wiki2.dovecot.org/Quota/Dict) does not mention DELETE being needed: Updated. From tss at iki.fi Tue Jul 17 16:05:09 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 16:05:09 +0300 Subject: [Dovecot] quota fs: Support for hardlimit quotas (where softquota is unset) In-Reply-To: <20120708094250.569c762d.jfriesse@gmail.com> References: <20120708094250.569c762d.jfriesse@gmail.com> Message-ID: <1342530309.19913.66.camel@hurina> On Sun, 2012-07-08 at 09:42 +0200, Jan Friesse wrote: > Hi, > included is patch which adds support for calculating fs quota from hardlimit where softlimit is unset. I'm usually not setting softlimit on my server causing dovecot work incorrectly (not reporting quota at all). Committed to v2.2: http://hg.dovecot.org/dovecot-2.2/rev/cdba6fec5ddf Seems like it could cause behavior to change for existing installations and maybe cause trouble, so I won't add it to v2.1. From tss at iki.fi Tue Jul 17 16:09:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 16:09:42 +0300 Subject: [Dovecot] Timeout (180s) while waiting for lock for transaction log file In-Reply-To: <4FFA9A14.1080607@um.es> References: <4FFA9A14.1080607@um.es> Message-ID: <1342530582.19913.69.camel@hurina> On Mon, 2012-07-09 at 10:45 +0200, Angel L. Mateo wrote: > We still have problems with our updated dovecot servers... The problem > now is that we are having errors like: > > Jul 9 10:22:02 myotis31 dovecot: lmtp(15431, ): Error: Timeout > (180s) while waiting for lock for transaction log file > /var/indexes//.INBOX/dovecot.index.log .. > maildir are in a nfs storage, but indexes are in local disk. I don't > know if the problem could be related with a some "specific" messages or > something specific to the user, because we are having this problem only > with a few users and the problem repeatedly happens when our mta tries > to deliver the queued message. If you can somewhat easily reproduce this for specific users, maybe you could strace -tt the lmtp process doing the delivery to see what is happening? Also, if the .log file is really locked for longer than 180secs, you should see a warning: Transaction log file %s was locked for %u seconds You could maybe also decrease the threshold for that warning: diff -r 56ef4e70b1a9 src/lib-index/mail-transaction-log-file.c --- a/src/lib-index/mail-transaction-log-file.c Tue Jul 17 15:44:36 2012 +0300 +++ b/src/lib-index/mail-transaction-log-file.c Tue Jul 17 16:09:28 2012 +0300 @@ -386,7 +386,7 @@ return; lock_time = time(NULL) - file->lock_created; - if (lock_time >= MAIL_TRANSCATION_LOG_LOCK_TIMEOUT) { + if (lock_time >= 30) { i_warning("Transaction log file %s was locked for %u seconds", file->filepath, lock_time); } From tss at iki.fi Tue Jul 17 16:21:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 16:21:19 +0300 Subject: [Dovecot] [PATCH] Interop problem with Cyrus SASL and GSSAPI In-Reply-To: <20120714181704.GB72986@anubis.morrow.me.uk> References: <20120714181704.GB72986@anubis.morrow.me.uk> Message-ID: <1342531279.19913.70.camel@hurina> On Sat, 2012-07-14 at 19:17 +0100, Ben Morrow wrote: > I have been trying to get a Postfix mail server using Dovecot SASL to > accept GSSAPI AUTH from another Postfix server using Cyrus SASL, and I > believe I have found a couple of bugs in Dovecot's GSSAPI > implementation. Committed: http://hg.dovecot.org/dovecot-2.1/rev/ced6a796f56d I also noticed another bug while checking that: http://hg.dovecot.org/dovecot-2.1/rev/35ae9569de5a From tim at tree-of-life.co.uk Tue Jul 17 17:14:15 2012 From: tim at tree-of-life.co.uk (Tim Dickson) Date: Tue, 17 Jul 2012 15:14:15 +0100 Subject: [Dovecot] files missed from install In-Reply-To: References: <4FF46CB3.9050901@tree-of-life.co.uk> Message-ID: <50057337.5080401@tree-of-life.co.uk> thanks very much, Tim On 17/07/2012 13:31, Timo Sirainen wrote: > On 4.7.2012, at 19.17, Tim Dickson wrote: > >> version: using dovecot 2.1.8 or 2.0.12 >> >> bug: installation - some files missed >> >> details: dovecot-openssl.cnf and mkcert.sh are not copied from the src/doc folder to the installation doc folder on installation (make install) >> both are referred to in the documentation and are useful. >> >> is there any chance of updating the build scripts to include these files? >> thanks, > I suppose they could be. http://hg.dovecot.org/dovecot-2.1/rev/d01a06d821cf > > From marcin at mejor.pl Tue Jul 17 17:15:05 2012 From: marcin at mejor.pl (=?UTF-8?B?TWFyY2luIE1pcm9zxYJhdw==?=) Date: Tue, 17 Jul 2012 16:15:05 +0200 Subject: [Dovecot] Dovecot-2.1.8 process imap crashed Message-ID: <50057369.4000103@mejor.pl> Hello! I've just upgraded dovecot from 2.0.18 to 2.1.8 and configured impac. I notice dovecot/imap throws segfault from time to time. I'm not sure i know what steps are needed to reproduce problem. I'm using gentoo, hardened-kernel-3.4.2. Namespaces (and settings for imapc) are keept in database so i'm not sure doveconf would be useful. If would be needed please let me know. Thanks Marcin Core was generated by `dovecot/imap'. Program terminated with signal 11, Segmentation fault. #0 0x000002d0fbf908a2 in mail_storage_set_internal_error (storage=0x0) at mail-storage.c:466 466 mail-storage.c: No such file or directory. in mail-storage.c (gdb) bt #0 0x000002d0fbf908a2 in mail_storage_set_internal_error (storage=0x0) at mail-storage.c:466 #1 0x000002d0fbf5f025 in imapc_noop_callback (reply=, context=) at imapc-storage.c:691 #2 0x000002d0fbf5f07a in imapc_idle_noop_callback (reply=, context=0x2b466158c0) at imapc-storage.c:713 #3 0x000002d0fbf6208a in imapc_connection_abort_commands (conn=0x2b46579c10, disconnected=, keep_retriable=) at imapc-connection.c:271 #4 0x000002d0fbf624be in imapc_connection_disconnect (conn=0x2b46579c10) at imapc-connection.c:371 #5 0x000002d0fbf636b9 in imapc_connection_deinit (_conn=0x2b46579b20) at imapc-connection.c:178 #6 0x000002d0fbf60113 in imapc_client_deinit (_client=0x2b4655b6e8) at imapc-client.c:111 #7 0x000002d0fbf5ea77 in imapc_storage_destroy (_storage=) at imapc-storage.c:275 #8 0x000002d0fbf90600 in mail_storage_unref (_storage=0x2b46558910) at mail-storage.c:417 #9 0x000002d0fbf8a259 in mail_namespace_free (ns=0x2b465588c0) at mail-namespace.c:37 #10 0x000002d0fbf8a784 in mail_namespaces_deinit (_namespaces=0x2b4654c9e8) at mail-namespace.c:437 #11 0x000002d0fbf95311 in mail_user_deinit_base (user=0x2b4654c970) at mail-user.c:29 #12 0x000002d0f940f658 in stats_user_deinit (user=0x2b4654c970) at stats-plugin.c:506 #13 0x000002d0fa86b619 in quota_user_deinit (user=0x2b4654c970) at quota-storage.c:412 #14 0x000002d0fbf95597 in mail_user_unref (_user=) at mail-user.c:153 #15 0x0000002b4461b58e in client_destroy (client=0x2b465891d0, reason=0x0) at imap-client.c:219 #16 0x0000002b44615552 in idle_client_input (ctx=) at cmd-idle.c:113 #17 0x000002d0fbce3767 in io_loop_call_io (io=0x2b46622ff0) at ioloop.c:379 #18 0x000002d0fbce469a in io_loop_handler_run (ioloop=) at ioloop-epoll.c:213 #19 0x000002d0fbce3054 in io_loop_run (ioloop=0x2b46525aa0) at ioloop.c:398 #20 0x000002d0fbccc446 in master_service_run (service=0x2b46525950, callback=) at master-service.c:543 #21 0x0000002b44624e76 in main (argc=1, argv=0x2b46525790) at main.c:389 (gdb) bt full #0 0x000002d0fbf908a2 in mail_storage_set_internal_error (storage=0x0) at mail-storage.c:466 str = 0x2b4651e840 "Internal error occurred. Refer to server log for more information. [2012-07-17 16:06:09]" #1 0x000002d0fbf5f025 in imapc_noop_callback (reply=, context=) at imapc-storage.c:691 storage = #2 0x000002d0fbf5f07a in imapc_idle_noop_callback (reply=, context=0x2b466158c0) at imapc-storage.c:713 mbox = 0x2b466158c0 #3 0x000002d0fbf6208a in imapc_connection_abort_commands (conn=0x2b46579c10, disconnected=, keep_retriable=) at imapc-connection.c:271 cmdp = 0x2b4651e808 cmd = 0x2b466116a0 tmp_array = {arr = {buffer = 0x2b4651e7c8, element_size = 8}, v = 0x2b4651e7c8, v_modifiable = 0x2b4651e7c8} reply = {state = IMAPC_COMMAND_STATE_DISCONNECTED, resp_text_key = 0x0, resp_text_value = 0x0, text_full = 0x2d0fbfe768e "Disconnected from server", text_without_resp = 0x2d0fbfe768e "Disconnected from server"} #4 0x000002d0fbf624be in imapc_connection_disconnect (conn=0x2b46579c10) at imapc-connection.c:371 reconnecting = false #5 0x000002d0fbf636b9 in imapc_connection_deinit (_conn=0x2b46579b20) at imapc-connection.c:178 No locals. #6 0x000002d0fbf60113 in imapc_client_deinit (_client=0x2b4655b6e8) at imapc-client.c:111 client = 0x2b4655c6e0 connp = 0x2b4655c840 __FUNCTION__ = "imapc_client_deinit" #7 0x000002d0fbf5ea77 in imapc_storage_destroy (_storage=) at imapc-storage.c:275 storage = #8 0x000002d0fbf90600 in mail_storage_unref (_storage=0x2b46558910) at mail-storage.c:417 storage = 0x2b4655b5e0 __FUNCTION__ = "mail_storage_unref" #9 0x000002d0fbf8a259 in mail_namespace_free (ns=0x2b465588c0) at mail-namespace.c:37 No locals. #10 0x000002d0fbf8a784 in mail_namespaces_deinit (_namespaces=0x2b4654c9e8) at mail-namespace.c:437 ns = next = 0x2b46550dd0 #11 0x000002d0fbf95311 in mail_user_deinit_base (user=0x2b4654c970) at mail-user.c:29 No locals. #12 0x000002d0f940f658 in stats_user_deinit (user=0x2b4654c970) at stats-plugin.c:506 suser = 0x2b4654e8c8 stats_conn = 0x2b46552170 __FUNCTION__ = "stats_user_deinit" #13 0x000002d0fa86b619 in quota_user_deinit (user=0x2b4654c970) at quota-storage.c:412 quser = 0x2b4654e890 quota_set = 0x2b46551190 #14 0x000002d0fbf95597 in mail_user_unref (_user=) at mail-user.c:153 user = 0x2b4654c970 __FUNCTION__ = "mail_user_unref" #15 0x0000002b4461b58e in client_destroy (client=0x2b465891d0, reason=0x0) at imap-client.c:219 cmd = __FUNCTION__ = "client_destroy" #16 0x0000002b44615552 in idle_client_input (ctx=) at cmd-idle.c:113 ---Type to continue, or q to quit--- client = 0x2b465891d0 #17 0x000002d0fbce3767 in io_loop_call_io (io=0x2b46622ff0) at ioloop.c:379 ioloop = 0x2b46525aa0 t_id = 2 #18 0x000002d0fbce469a in io_loop_handler_run (ioloop=) at ioloop-epoll.c:213 ctx = 0x2b46525e10 events = event = 0x2b46525e80 list = 0x2b465a80a0 io = tv = {tv_sec = 5, tv_usec = 999824} msecs = ret = 1 i = j = call = #19 0x000002d0fbce3054 in io_loop_run (ioloop=0x2b46525aa0) at ioloop.c:398 No locals. #20 0x000002d0fbccc446 in master_service_run (service=0x2b46525950, callback=) at master-service.c:543 No locals. #21 0x0000002b44624e76 in main (argc=1, argv=0x2b46525790) at main.c:389 set_roots = {0x2b4482ad60, 0x0} login_set = {auth_socket_path = 0x2b4651d498 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x2b44624804 , failure_callback = 0x2b446244fd } service_flags = storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = From tss at iki.fi Tue Jul 17 18:39:19 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 17 Jul 2012 18:39:19 +0300 Subject: [Dovecot] Dovecot-2.1.8 process imap crashed In-Reply-To: <50057369.4000103@mejor.pl> References: <50057369.4000103@mejor.pl> Message-ID: <602C5811-214D-441E-8E17-F3FD9993301B@iki.fi> On 17.7.2012, at 17.15, Marcin Miros?aw wrote: > Hello! > I've just upgraded dovecot from 2.0.18 to 2.1.8 and configured impac. I > notice dovecot/imap throws segfault from time to time. I'm not sure i > know what steps are needed to reproduce problem. .. > #3 0x000002d0fbf6208a in imapc_connection_abort_commands > (conn=0x2b46579c10, disconnected=, > keep_retriable=) at imapc-connection.c:271 > #4 0x000002d0fbf624be in imapc_connection_disconnect > (conn=0x2b46579c10) at imapc-connection.c:371 .. > #15 0x0000002b4461b58e in client_destroy (client=0x2b465891d0, > reason=0x0) at imap-client.c:219 > #16 0x0000002b44615552 in idle_client_input (ctx=) at > cmd-idle.c:113 This happened when the client was IDLEing and the connection got closed. It's definitely a bug, but I haven't so far bothered to look into it much because the end result from the client's point of view is the same anyway (getting disconnected). From rblayzor.bulk at inoc.net Tue Jul 17 19:19:59 2012 From: rblayzor.bulk at inoc.net (Robert Blayzor) Date: Tue, 17 Jul 2012 12:19:59 -0400 Subject: [Dovecot] Selective TLS per local IP In-Reply-To: <41303680-06C4-41FB-AA9E-40C2945FEA13@iki.fi> References: <30D71C3F-4356-4B09-A8F3-6E83501B2C27@inoc.net> <41303680-06C4-41FB-AA9E-40C2945FEA13@iki.fi> Message-ID: <58C9EFC8-7001-4ACD-A141-4F070651773C@inoc.net> On Jul 17, 2012, at 8:28 AM, Timo Sirainen wrote: >> No, they do not. It would be nice if it did. Or at least some way to disable TLS offering/advertisement if disabling the SSL socket is not possible. > > Here: http://hg.dovecot.org/dovecot-2.1/rev/258c2e231357 Works perfectly, thanks! -- Robert Blayzor INOC, LLC rblayzor at inoc.net http://www.inoc.net/~rblayzor/ From lazy404 at gmail.com Wed Jul 18 00:07:44 2012 From: lazy404 at gmail.com (Lazy) Date: Tue, 17 Jul 2012 23:07:44 +0200 Subject: [Dovecot] outlook and redirect sieve In-Reply-To: <50042D92.1020700@rename-it.nl> References: <50042D92.1020700@rename-it.nl> Message-ID: 2012/7/16 Stephan Bosch : > On 7/16/2012 5:00 PM, Lazy wrote: >> >> 2012/7/16 Lazy : >> >> >> i have got it >> >> if ( recipient != NULL ) >> rfc2822_header_field_write(f, >> "X-Sieve-Redirected-From", recipient); >> >> was adding crlf in the headers, and it made whole email inconsistent >> (when I have removed crlf filter before) > > > Could you explain this a little further? Is this a bug? it turned out it was a qmail issue, there is a patch for it already, http://opensource.sf-tec.de/qmail/qmail-remote-CRLF.diff sorry for the noise Regards, Michal Grzedzicki From fxmulder at gmail.com Wed Jul 18 02:04:50 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 17 Jul 2012 17:04:50 -0600 Subject: [Dovecot] Different LDAP filters for different protocols Message-ID: Is it possible to query LDAP differently for different protocols? I am using dovecot for both pop3/imap and lmtp deliveries and there are different enable flags in ldap for them. I tried wrapping user_filter inside the protocols in the file dovecot-ldap.conf.ext as shown below but this didn't seem to work. protocol lmtp { user_filter = (&(objectClass=posixAccount)(cn=%u)(active=1)(msmtp=1)) } protocol imap { user_filter = (&(objectClass=posixAccount)(cn=%u)(active=1)(mpop=1)) } protocol pop3 { user_filter = (&(objectClass=posixAccount)(cn=%u)(active=1)(mpop=1)) } From fxmulder at gmail.com Wed Jul 18 02:18:39 2012 From: fxmulder at gmail.com (James Devine) Date: Tue, 17 Jul 2012 17:18:39 -0600 Subject: [Dovecot] Different LDAP filters for different protocols In-Reply-To: References: Message-ID: nevermind I think I got it, wrapping the ldap userdb in the protocols and making a copy of the ldap configuration files seems to work On Tue, Jul 17, 2012 at 5:04 PM, James Devine wrote: > Is it possible to query LDAP differently for different protocols? I am > using dovecot for both pop3/imap and lmtp deliveries and there are > different enable flags in ldap for them. I tried wrapping user_filter > inside the protocols in the file dovecot-ldap.conf.ext as shown below but > this didn't seem to work. > > protocol lmtp { > user_filter = > (&(objectClass=posixAccount)(cn=%u)(active=1)(msmtp=1)) > } > > protocol imap { > user_filter = > (&(objectClass=posixAccount)(cn=%u)(active=1)(mpop=1)) > } > > protocol pop3 { > user_filter = > (&(objectClass=posixAccount)(cn=%u)(active=1)(mpop=1)) > } > > From lazy404 at gmail.com Wed Jul 18 04:33:07 2012 From: lazy404 at gmail.com (Lazy) Date: Wed, 18 Jul 2012 03:33:07 +0200 Subject: [Dovecot] dovecot-lda setting quota rules Message-ID: Hi, in my setup dovecot-lda reads user and home from environment, and dosn't perform userdb lookup Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: Effective uid=300, gid=300, home=/vmail/la/lazytest_test Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: Quota root: name=User quota backend=maildir args= Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: Quota warning: bytes=0 (95%) messages=0 reverse=no command=quota-warning 95 lazytest_test Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: Quota warning: bytes=0 (80%) messages=0 reverse=no command=quota-warning 80 lazytest_test Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: Namespace inbox: type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes l ocation=maildir:~/Maildir:INDEX=/var/dovecot_indexes/vmail/la/lazytest_test Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: maildir++: root=/vmail/la/lazytest_test/Maildir, index=/var/dovecot_indexes/vmail/la/lazytest_tes t, control=, inbox=/vmail/la/lazytest_test/Maildir, alt= Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: userdb lookup skipped, username taken from USER environment Jul 18 03:09:47 thebe2 dovecot: lda(lazytest_test): Debug: Quota root: name=User quota backend=maildir args= I wan't it to stay that way. The problem is that it dosn't update the quota rules in Maildir when quota is changed in userdb. I have the rules avaiable in qmail-lda's environment, but I need an idea on how to feed the quota rule string to quota plugin without doing userdb lookup. What whould be the best way to do this, maybe in plugins/quota/quota.c:quota_root_add_rules() ? Regards, Michal Grzedzicki From Jim at themailshack.com Wed Jul 18 08:09:00 2012 From: Jim at themailshack.com (Jim) Date: Wed, 18 Jul 2012 00:09:00 -0500 Subject: [Dovecot] Compile issue Dovecot 2.1.8, OS X 10.6.x In-Reply-To: <961F8926-34D0-476B-9B92-202340B73DF7@themailshack.com> References: <961F8926-34D0-476B-9B92-202340B73DF7@themailshack.com> Message-ID: <2CAF8DF4-1A95-4C79-AC18-8DCC7E2F6D40@themailshack.com> Hey all, I'm still stumped here. No suggestions from anyone on anything I can look at? Has anyone successfully compiled the 2.1.x branch on Mac OS X 10.6.x? Jim On Jul 16, 2012, at 8:10 PM, Jim wrote: > I just did tried building Dovecot 2.1.0, same issue. Dovecot 2.0.21 was able to build without issues, so this problem seems to be with the entire 2.1.x branch. > > > Jim > > On Jul 16, 2012, at 4:25 PM, Jim wrote: > >> Hey all. I'm running Mac OS X 10.6.4, and had Dovecot 2.0.7 installed built from source. I'm now wanting to upgrade to Dovecot 2.1.8, but I can't get MAKE to finish without errors: >> >> Undefined symbols: >> "_libiconv_open", referenced from: >> _charset_to_utf8_begin in libcharset.a(charset-iconv.o) >> "_libiconv", referenced from: >> _charset_to_utf8_reset in libcharset.a(charset-iconv.o) >> _charset_to_utf8 in libcharset.a(charset-iconv.o) >> "_libiconv_close", referenced from: >> _charset_to_utf8_end in libcharset.a(charset-iconv.o) >> ld: symbol(s) not found >> collect2: ld returned 1 exit status >> make[3]: *** [libdovecot.la] Error 1 >> make[2]: *** [all-recursive] Error 1 >> make[1]: *** [all-recursive] Error 1 >> make: *** [all] Error 2 > From tss at iki.fi Wed Jul 18 08:15:13 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 18 Jul 2012 08:15:13 +0300 Subject: [Dovecot] Compile issue Dovecot 2.1.8, OS X 10.6.x In-Reply-To: <2CAF8DF4-1A95-4C79-AC18-8DCC7E2F6D40@themailshack.com> References: <961F8926-34D0-476B-9B92-202340B73DF7@themailshack.com> <2CAF8DF4-1A95-4C79-AC18-8DCC7E2F6D40@themailshack.com> Message-ID: I always use: CPPFLAGS='-I/opt/local/include/postgresql85 -I/opt/local/include' \ LDFLAGS="-L/opt/local/lib/postgresql85 -L/opt/local/lib" \ ./configure Because otherwise system iconv conflicts with macports iconv. But it has been this way in v2.0 as well. On 18.7.2012, at 8.09, Jim wrote: > Hey all, I'm still stumped here. No suggestions from anyone on anything I can look at? Has anyone successfully compiled the 2.1.x branch on Mac OS X 10.6.x? > > > Jim > > > On Jul 16, 2012, at 8:10 PM, Jim wrote: > >> I just did tried building Dovecot 2.1.0, same issue. Dovecot 2.0.21 was able to build without issues, so this problem seems to be with the entire 2.1.x branch. >> >> >> Jim >> >> On Jul 16, 2012, at 4:25 PM, Jim wrote: >> >>> Hey all. I'm running Mac OS X 10.6.4, and had Dovecot 2.0.7 installed built from source. I'm now wanting to upgrade to Dovecot 2.1.8, but I can't get MAKE to finish without errors: >>> >>> Undefined symbols: >>> "_libiconv_open", referenced from: >>> _charset_to_utf8_begin in libcharset.a(charset-iconv.o) >>> "_libiconv", referenced from: >>> _charset_to_utf8_reset in libcharset.a(charset-iconv.o) >>> _charset_to_utf8 in libcharset.a(charset-iconv.o) >>> "_libiconv_close", referenced from: >>> _charset_to_utf8_end in libcharset.a(charset-iconv.o) >>> ld: symbol(s) not found >>> collect2: ld returned 1 exit status >>> make[3]: *** [libdovecot.la] Error 1 >>> make[2]: *** [all-recursive] Error 1 >>> make[1]: *** [all-recursive] Error 1 >>> make: *** [all] Error 2 >> > From Jim at themailshack.com Wed Jul 18 08:47:10 2012 From: Jim at themailshack.com (Jim) Date: Wed, 18 Jul 2012 00:47:10 -0500 Subject: [Dovecot] Compile issue Dovecot 2.1.8, OS X 10.6.x In-Reply-To: References: <961F8926-34D0-476B-9B92-202340B73DF7@themailshack.com> <2CAF8DF4-1A95-4C79-AC18-8DCC7E2F6D40@themailshack.com> Message-ID: Hey Timo. I tried this just now, and that did the trick! I've updated my documentation here to include these additional parameters. Thanks so much! Jim On Jul 18, 2012, at 12:15 AM, Timo Sirainen wrote: > I always use: > > CPPFLAGS='-I/opt/local/include/postgresql85 -I/opt/local/include' \ > LDFLAGS="-L/opt/local/lib/postgresql85 -L/opt/local/lib" \ > ./configure > > Because otherwise system iconv conflicts with macports iconv. But it has been this way in v2.0 as well. > > On 18.7.2012, at 8.09, Jim wrote: > >> Hey all, I'm still stumped here. No suggestions from anyone on anything I can look at? Has anyone successfully compiled the 2.1.x branch on Mac OS X 10.6.x? >> >> >> Jim >> >> >> On Jul 16, 2012, at 8:10 PM, Jim wrote: >> >>> I just did tried building Dovecot 2.1.0, same issue. Dovecot 2.0.21 was able to build without issues, so this problem seems to be with the entire 2.1.x branch. >>> >>> >>> Jim >>> >>> On Jul 16, 2012, at 4:25 PM, Jim wrote: >>> >>>> Hey all. I'm running Mac OS X 10.6.4, and had Dovecot 2.0.7 installed built from source. I'm now wanting to upgrade to Dovecot 2.1.8, but I can't get MAKE to finish without errors: >>>> >>>> Undefined symbols: >>>> "_libiconv_open", referenced from: >>>> _charset_to_utf8_begin in libcharset.a(charset-iconv.o) >>>> "_libiconv", referenced from: >>>> _charset_to_utf8_reset in libcharset.a(charset-iconv.o) >>>> _charset_to_utf8 in libcharset.a(charset-iconv.o) >>>> "_libiconv_close", referenced from: >>>> _charset_to_utf8_end in libcharset.a(charset-iconv.o) >>>> ld: symbol(s) not found >>>> collect2: ld returned 1 exit status >>>> make[3]: *** [libdovecot.la] Error 1 >>>> make[2]: *** [all-recursive] Error 1 >>>> make[1]: *** [all-recursive] Error 1 >>>> make: *** [all] Error 2 >>> >> > From kamath at moltingpenguin.com Wed Jul 18 10:51:47 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Wed, 18 Jul 2012 00:51:47 -0700 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <592BA186-388E-442B-86B8-4A1B36B9BBE6@moultingpenguin.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> <5003F09F.40908@Media-Brokers.com> <50053C0A.5090208@Media-Brokers.com> <592BA186-388E-442B-86B8-4A1B36B9BBE6@moultingpenguin.com> Message-ID: <7319F6C7-948B-40F6-ACE4-3F8D87CA9072@moltingpenguin.com> On Jul 17, 2012, at 3:18 AM, Charles Marcus wrote: >> I really thought it would be a simple "here's what you missed", since >> the logs are relatively uninformative... > > The point is, what may appear to be uninformative to you may tell Timo (or someone else who knows what to look for) all they need to know to pinpoint the problem. Whilst I would normally agree, at the same time I've had a fair amount of experience with dovecot and sendmail (been running them for over 10 years). Pigeonhole is new to me, however. . . > One thing I did notice is that you are sending this mail from an internal IP... since most people use very different checks for such mail (bypassing many/most checks), maybe you should be testing from an external address (one that comes in from outside your network)? So I have a DMZ that houses the mail server for my domain, which is where the private addresses are. Connections are passed through the OpenBSD router to the mail server. So testing from an external address would be essentially the same (since the connection to the SMTP server talks to the sendmail process as the /usr/lib/sendmail process). Regardless, my original tests were sending mail from the Mac off-network. > Also, it is always recommended when setting up a new server to start with the latest version... for 2.0.x that would be 2.0.19, but the latest is 2.1.8... When I set it up, 2.0.13 _was_ the latest. :-) I was hoping to avoid the pain of recompiling dovecot and pigeonhole. Not that big a deal, I guess. . . >> lda_original_recipient_header = X-Original-To >> lmtp_save_to_detail_mailbox = yes > > Ok, I think this is the problem... > > Dovecot's LMTP currently does NOT preserve the x-original-to header... in fact, brought this up recently (I was one of the ones interested in Timo adding support for it) on this (and the postfix, which is what I use) on this very list: > > http://www.dovecot.org/list/dovecot/2012-April/065075.html > > Apparently, after Timo discussed this on the postfix list, he determined that it would not be a very significant change for him to make to add support for this header in LMTP and that he would get to it sooner than later, but as of now, it has not been added. Actually, I'm not using lmtp. I whacked the lmtp_save_to_detail_mailbox in an effort to see if that was the issue before I realized I don't have it turned on. dovecot.conf has: protocols = imap though in 20-managesieve.conf I have: protocols = $protocols sieve I just turned it off and restarted dovecot (though lmtp was not running) and re-ran the test with the same result. So, I think what's going on is pretty straightforward. The local mailer definition in the sendmail.cf file is: Mlocal, P=/usr/local/libexec/dovecot/dovecot-lda, F=lsDFMAw5:/|@qSPhn9, S=EnvFromL/HdrFromL, R=EnvToL/HdrToL, T=DNS/RFC822/X-Unix, A=/usr/local/libexec/dovecot/dovecot-lda -d $u This is more or less right out of the wiki. Since dovecot-lda is passed the -d option, it's using that (and only that) for delivery. Since there's no X-Original-To: header, it doesn't really have any option (since the non-envelope-to address might not exist, or point to something other than the end user's mailbox, etc). It would appear that I have two options: 1) Figure out what macro(s) is(are) valid to pass to dovecot-lda with the -a option in the sendmail config file (sendmail.cf). 2) Get sendmail to put "X-Original-To:
" in the header. Both would appear to be a sendmail issue, not a dovecot/pigeonhole issue. Everything I find on the web seems to center around how to use plus addressing to accept mail from multiple email address and filter it into a single address for delivery, only to have the local delivery agent break it back out again. That's not my issue. :-( Sean Sean From michael at think-for-yourself.org Wed Jul 18 11:40:06 2012 From: michael at think-for-yourself.org (Michael Wessel) Date: Wed, 18 Jul 2012 01:40:06 -0700 Subject: [Dovecot] expire-tool stops on first box with nothing to expire in it In-Reply-To: <50025FD9.6000701@schetterer.org> References: <50023A36.4070705@think-for-yourself.org> <50025FD9.6000701@schetterer.org> Message-ID: <50067666.2090208@think-for-yourself.org> On 7/14/2012 11:14 PM, Robert Schetterer wrote: > 1.2.11 is outdated you should upgrade 2.1.8 and try again Thanks. I knew the version was old but I wasn't quite ready for the upgrade. Anyway, I got ready, sat down and upgraded to 2.1.8 this weekend and have it all running again now. Expiring however still behaves the same as far as I can tell. The output is different now using doveadm but the behavior seems the same or at least very similar. When I run for example: doveadm -D search -A mailbox Trash savedbefore 60d > /tmp/discard (the > /tmp/discard is only there to send all the listed messages into the file to make the output more comprehensive) the results always end in something like: doveadm(user at domain.org): Debug: expire: Stopping iteration on key shared/expire/nextuser at domain.org/Trash (1340238662 > 1337415398) If I use different savedbefore values, it stops in different places, but always the same with the same value. If I set the value way down to 3 it only goes through about 70 boxes before it stops. Yet there are over 1000 in the expire database. And if I run the same command on a specific user (one that's not in the result set of -A) rather than -A I do get a list of messages. So there are definitely users with messages that should be returned as part of this but are not. Any ideas on what's wrong? Just in case my doveconf -n is below. Michael # 2.1.8: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-92.el5 x86_64 CentOS release 5.2 (Final) ext3 auth_master_user_separator = * base_dir = /var/run/dovecot dict { expire = mysql:/usr/local/etc/dovecot-dict-expire.conf } disable_plaintext_auth = no info_log_path = /var/log/dovecot.log listen = 10.254.100.254,localhost,10.254.100.253 log_path = /var/log/dovecot.err mail_gid = 5000 mail_location = maildir:/usr/local/mail/%1u/%1.1u/%u mail_plugins = " expire quota" mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /usr/local/etc/restrict-users driver = passwd-file } passdb { args = /usr/local/etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { driver = shadow } passdb { args = /usr/local/etc/dovecot-ldap.conf driver = ldap } plugin { autocreate = Trash autocreate2 = Spam autocreate3 = Drafts autocreate4 = Sent autosubscribe = Trash autosubscribe2 = Spam autosubscribe3 = Drafts autosubscribe4 = Sent expire = Spam expire2 = Trash expire3 = Trash/* expire_dict = proxy::expire quota = maildir:User quota quota_warning = storage=80%% quota-warning 80 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = storage=90%% quota-warning 90 %u quota_warning4 = storage=95%% quota-warning 95 %u sieve = /usr/local/users/%u/.dovecot.sieve sieve_dir = /usr/local/users/%u/sieve } protocols = imap pop3 sieve service auth { unix_listener auth-userdb { mode = 0600 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { executable = /usr/local/libexec/dovecot/imap-login process_limit = 384 process_min_avail = 5 service_count = 1 } service imap-postlogin { executable = script-login /usr/local/bin/last_login.sh user = $default_internal_user } service imap { executable = /usr/local/libexec/dovecot/imap imap-postlogin process_limit = 1024 } service managesieve-login { process_min_avail = 5 service_count = 1 } service pop3-login { executable = /usr/local/libexec/dovecot/pop3-login process_limit = 384 process_min_avail = 5 service_count = 1 } service pop3 { executable = /usr/local/libexec/dovecot/pop3 process_limit = 1024 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = dovecot } ssl_cert = References: <50057369.4000103@mejor.pl> <602C5811-214D-441E-8E17-F3FD9993301B@iki.fi> Message-ID: <50067D6B.9050401@mejor.pl> W dniu 17.07.2012 17:39, Timo Sirainen pisze: > This happened when the client was IDLEing and the connection got closed. It's definitely a bug, but I haven't so far bothered to look into it much because the end result from the client's point of view is the same anyway (getting disconnected). Hi! Thanks for information, good to know it doesn't any hurt to client. Regards, Marcin From micha at krausam.de Wed Jul 18 15:49:24 2012 From: micha at krausam.de (Micha Krause) Date: Wed, 18 Jul 2012 14:49:24 +0200 Subject: [Dovecot] FETCH Error Message-ID: <5006B0D4.9010602@krausam.de> Hi, I have the following problem: Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: FETCH [] for mailbox INBOX.Sent UID 166 got too little data: 120860 vs 120867 Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: Maildir filename has wrong W value: /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003,W=284007:2,S Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: Corrupted index cache file /var/mail/mailboxes/users/micha/.Sent/dovecot.index.cache: Broken virtual size for mail UID 166 so far i tried deleting: /var/mail/mailboxes/users/micha/.Sent/dovecot.index.cache /var/mail/mailboxes/users/micha/.Sent/dovecot.index /var/mail/mailboxes/users/micha/.Sent/dovecot.index.log I tried to renaming: /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003,W=284007:2,S to /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003:2,S I tried doveadm: doveadm force-resync -u micha INBOX.Sent Dovecot version is: 2.0.9 Micha Krause From list at airstreamcomm.net Wed Jul 18 19:24:20 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 18 Jul 2012 11:24:20 -0500 Subject: [Dovecot] Dsync replication In-Reply-To: <43111E61-9EA3-4D18-B55B-C0F40DD891BE@iki.fi> References: <4FF5C09F.2060001@airstreamcomm.net> <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> <4FF74A76.6010605@airstreamcomm.net> <43111E61-9EA3-4D18-B55B-C0F40DD891BE@iki.fi> Message-ID: <5006E334.7010903@airstreamcomm.net> On 7/6/12 9:56 PM, Timo Sirainen wrote: > On 6.7.2012, at 23.28, list at airstreamcomm.net wrote: > >> Thanks, that certainly helps identify the configuration options. However I am more concerned about the experiences of others who have actually used the replication. What is the rate of change on your mail cluster, how many concurrent users do you support with replication enabled, do you use synchronous or asynchronous replication, are you using it in an active/active or active/passive state, is it possible to have a cluster with multiple servers at each site hosting the same mail data, does dysnc replication scale well (10,000 -> 100,000 -> 1,000,000 users)? Just trying to get a good feel for whether dsync replication is capable of handling the use case I am proposing before investing too much time in testing it. > I wouldn't use it for large systems yet. It is still pretty inefficient. v2.2 will have a redesigned dsync that can do incremental syncs much faster and with less bandwidth. > > Anyway, in my small installation I'm using it in active-active mode and it works well enough. I've even configured my clients intentionally so that they use different servers. > > Timo, Does dsync replication only work between two hosts? In my scenario I would have two sites with X number of nodes at each with an NFS backend for each site. For this example lets say I have site A with two nodes that mount one NFS share, and site B with two nodes that mount one NFS share. Is it possible to implement dsync replication between these two clusters of nodes? From list at airstreamcomm.net Wed Jul 18 19:32:26 2012 From: list at airstreamcomm.net (list at airstreamcomm.net) Date: Wed, 18 Jul 2012 11:32:26 -0500 Subject: [Dovecot] MySQL dict issues In-Reply-To: References: <4FFDC1A0.6040603@airstreamcomm.net> Message-ID: <5006E51A.1010309@airstreamcomm.net> On 7/17/12 7:59 AM, Timo Sirainen wrote: > On 11.7.2012, at 21.10, list at airstreamcomm.net wrote: > >> Testing using the mysql dict for quota storage (dovecot-2.0.12-2_127.el5), and ran into a couple issues. First is a permissions issue: >> >> dovecot: lmtp(26786, dude at test.tld): Error: net_connect_unix(/var/run/dovecot/dict) failed: Permission denied (euid=5000() egid=5000() missing +r perm: /var/run/dovecot/dict, euid is not dir owner) >> >> The file /var/run/dovecot/dict is listed with srw------- perms. I changes the perms to srw----rwx and it works, but I would like to figure out how to configure properly to run with the uid and gid 5000 as this is the user for virtual mail. > http://wiki2.dovecot.org/Dict explains this. It does indeed, thanks. > >> The second issue is that two identical rows are created in the mysql database for each user quota. The next message delivered updates both rows. > Identical? Or perhaps one is the number of bytes and the other is the number of messages? There were two rows being created for each username. My problem was that I forgot to create a PK on the username field, so there must be two operations on the database that create/update user information by dovecot that require that PK to keep it to one row. > >> The last issue might be a documentation omission. Dovecot warned me that the mysql user did not have DELETE privileges on the quota table: >> >> dovecot: dict: Error: sql dict: commit failed: DELETE command denied to user 'user'@'a.b.c.d' for table 'domain_user_quotas' > This is done when recalculating quota. > >> The quota dict documentation (at http://wiki2.dovecot.org/Quota/Dict) does not mention DELETE being needed: > Updated. Thanks! From Nick at 5sg.eu Thu Jul 19 00:39:13 2012 From: Nick at 5sg.eu (Nick at 5sg.eu) Date: Wed, 18 Jul 2012 17:39:13 -0400 Subject: [Dovecot] Converting from Maildir format ? Message-ID: <50072D01.1000905@5sg.eu> Hi again, I'm still trying to find out if anyone knows the correct way to either switch the existing Maildir multiple file format to the single file format I believe known as MBox. Failing that, though it'll mean a LOT of work for me to manually convert all of the existing Maildirs, how do I get dovecot to work in future in MBox single file format. I tried entering 'mail_location = mbox:~/mail:INBOX=/%h/%u' in the 'dovecot.conf', but it still saves each message in a file of its own in the /new /cur /tmp directories ?? Thanks, Nick. . . . . -- _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ . . . . . . . . ...---... (Stamp Out Spam and Save Our Servers) http://sosNow.ws -=[ do you know morse code ? ]=- . . . . . . . . _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ From xxyokc at gmail.com Thu Jul 19 01:21:53 2012 From: xxyokc at gmail.com (David Drexler) Date: Wed, 18 Jul 2012 17:21:53 -0500 Subject: [Dovecot] doesn't work at all Message-ID: Hi, I can't seem to get dovecot to work. When I connect with thunderbird, tbird complains that it can't find the settings for my email. When I connect with evolution, it seems to go through all the motions but it doesn't pick up the waiting messages. Those messages are marked Status: O s if it had read them. The log file for tbird looks like this: Jul 18 17:03:21 why dovecot: imap-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: imap-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: imap-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: pop3-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: pop3-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: pop3-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: imap-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: pop3-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: imap-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: pop3-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 Jul 18 17:03:22 why dovecot: pop3-login: Aborted login (no auth attempts): rip=68.12.110.134, lip=184.176.6.227 The log file with evolution is: Jul 17 19:36:43 why dovecot: imap-login: Login: user=, method=PLAIN, rip=68. 12.110.134, lip=184.176.6.227, mpid=22717, TLS Jul 17 19:36:43 why dovecot: imap(ddrex): Debug: Effective uid=500, gid=500, home=/ home/ddrex Jul 17 19:36:43 why dovecot: imap(ddrex): Debug: fs: root=/home/ddrex/mail, index=, control=, inbox=/var/mail/ddrex Jul 17 19:46:17 why dovecot: imap(ddrex): Error: Next message unexpectedly corrupte d in mbox file /var/mail/ddrex at 1504 My conf file: # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.23.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) base_dir = /var/run/dovecot/ imap_client_workarounds = "delay-newmail " listen = * mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_privileged_group = mail mbox_very_dirty_syncs = yes mbox_write_locks = fcntl passdb { driver = pam } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh protocols = imap pop3 service imap-login { vsz_limit = 64 M } service pop3-login { vsz_limit = 64 M } ssl_cert = References: Message-ID: I forgot to mention, it passes the tests at http://wiki2.dovecot.org/TestInstallation From jtam.home at gmail.com Thu Jul 19 04:24:07 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 18 Jul 2012 18:24:07 -0700 (PDT) Subject: [Dovecot] expire-tool stops on first box with nothing to expire in it In-Reply-To: References: Message-ID: Michael Wessel writes: > When I run for example: > > doveadm -D search -A mailbox Trash savedbefore 60d > /tmp/discard I previously observed that the "date.saved" field (tested by savedbefore) seems to be clustered around particular timestamps. Dumping out this field using doveadm -ftab fetch -A \ "mailbox date.saved" \ \( mailbox Trash OR mailbox Junk OR mailbox "Deleted Items" \) it seems the timestamps do not actually correspond to when the client moves messages to their trash folder, but rather, to the time I query for it i.e. a query instantiates this value. This timestamp is then retained. In practice, if this query is done on a periodic interval (e.g. daily), the date.saved will be roughly correct, and is usable as an age test for expungement. Is there a way to force this value to be cached when the client actually saves or moves a message? Joseph Tam From dmalolepszy at optusnet.com.au Thu Jul 19 08:09:11 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Thu, 19 Jul 2012 15:09:11 +1000 Subject: [Dovecot] SQLite dovecot query caching In-Reply-To: <9D7331EF-9E49-4ABD-A941-B0874BDA012B@iki.fi> References: <4FE5FEB4.2050801@optusnet.com.au> <1340473147.5967.89.camel@hurina> <4FE608E8.6090106@optusnet.com.au> <1340477828.5967.91.camel@hurina> <4FFFB262.7040409@optusnet.com.au> <9D7331EF-9E49-4ABD-A941-B0874BDA012B@iki.fi> Message-ID: <50079677.90604@optusnet.com.au> On 17/07/12 21:28, Timo Sirainen wrote: > > The simplest solution would be to just not put username to cache, but I'm worried it might break something else.. So for now I don't think I'm going to change it. > Fair enough, last thing I want is you breaking Dovecot, I was just hoping I was doing something wrong. I have it working by including the username in the sqlite query so it is included part of the cache key, and will just up the cache ttl to something like 24 hours and see how things go, thanks again Timo! From stano at websupport.sk Wed Jul 18 17:48:24 2012 From: stano at websupport.sk (Pavel Stano) Date: Wed, 18 Jul 2012 16:48:24 +0200 Subject: [Dovecot] [PATCH] Pop3/imap ordering in courier-dovecot-migrate.pl Message-ID: <5006CCB8.6050701@websupport.sk> Hi, current version of courier migration script courier-dovecot-migrate.pl preserve order of POP3 UIDLs, but it keep IMAP UIDs not in order which cause problems with clients using IMAP along with POP3 in some cases. Attached patch use MAIL_FETCH_POP3_ORDER feature added in version 2.0.13 to keep POP3 and IMAP UID[L]s in order. -- [ Ohodnotte kvalitu mailu: http://nicereply.com/websupport/4afafd34/ ] Pavel Stano | admin team http://WebSupport.sk *** TAKE AND ENJOY *** -------------- next part -------------- A non-text attachment was scrubbed... Name: courier-dovecot-migrate.patch Type: text/x-patch Size: 2289 bytes Desc: not available URL: From stan at hardwarefreak.com Thu Jul 19 08:31:01 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Jul 2012 00:31:01 -0500 Subject: [Dovecot] Converting from Maildir format ? In-Reply-To: <50072D01.1000905@5sg.eu> References: <50072D01.1000905@5sg.eu> Message-ID: <50079B95.8020100@hardwarefreak.com> On 7/18/2012 4:39 PM, Nick at 5sg.eu wrote: > Hi again, > I'm still trying to find out if anyone knows the correct > way to either switch the existing Maildir multiple file format to > the single file format I believe known as MBox. > Failing that, though it'll mean a LOT of work for me to manually > convert all of the existing Maildirs, how do I get dovecot to > work in future in MBox single file format. > > I tried entering 'mail_location = mbox:~/mail:INBOX=/%h/%u' in > the 'dovecot.conf', but it still saves each message in a file of > its own in the /new /cur /tmp directories ?? Did you restart dovecot after the mailbox_location change? Show 'dovecot -n' output. -- Stan From stan at hardwarefreak.com Thu Jul 19 08:53:17 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Thu, 19 Jul 2012 00:53:17 -0500 Subject: [Dovecot] doesn't work at all In-Reply-To: References: Message-ID: <5007A0CD.1030309@hardwarefreak.com> On 7/18/2012 5:21 PM, David Drexler wrote: > Hi, > > I can't seem to get dovecot to work. When I connect with thunderbird, > tbird complains that it can't find the settings for my email. When I > connect with evolution, it seems to go through all the motions but it > doesn't pick up the waiting messages. Those messages are marked Status: O > s if it had read them. > > The log file for tbird looks like this: > > Jul 18 17:03:21 why dovecot: imap-login: Aborted login (no auth attempts): > rip=68.12.110.134, lip=184.176.6.227 Tbird isn't attempting to authenticate. Match your Tbrid settings to those you've configured in Dovecot auth. > Jul 17 19:46:17 why dovecot: imap(ddrex): Error: Next message unexpectedly > corrupte > d in mbox file /var/mail/ddrex at 1504 The user mbox file is corrupted. Fix that and Ev may work. Usually the problem is that a single "F" has been prepended to "From", which is the delimiter between emails in the file. If that's the case you can fix it with vi/vim etc, by deleting the leading "F". Search for "FFrom", delete leading F, save file. If it's beyond this, and you're not concerned with the contents of the file, simply delete the mbox file and let Dovecot create a new one. > My conf file: > > # 2.0.9: /etc/dovecot/dovecot.conf > # OS: Linux 2.6.32-220.23.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) > base_dir = /var/run/dovecot/ > imap_client_workarounds = "delay-newmail " Why are you using this workaround? It's not needed for Tbird or Evolution. > listen = * > mail_debug = yes > mail_location = mbox:~/mail:INBOX=/var/mail/%u > mail_privileged_group = mail > mbox_very_dirty_syncs = yes > mbox_write_locks = fcntl > passdb { > driver = pam > } > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > protocols = imap pop3 > service imap-login { > vsz_limit = 64 M > } > service pop3-login { > vsz_limit = 64 M > } > ssl_cert = ssl_key = userdb { > driver = passwd > } > -------------------------------------------------------------------- > > Thanks for any clues. > > dd > From kamath at moltingpenguin.com Thu Jul 19 09:57:22 2012 From: kamath at moltingpenguin.com (Sean Kamath) Date: Wed, 18 Jul 2012 23:57:22 -0700 Subject: [Dovecot] Issues with Plus addressing and detail part of subaddress. In-Reply-To: <7319F6C7-948B-40F6-ACE4-3F8D87CA9072@moltingpenguin.com> References: <7109BC12-505E-46E0-AE2E-91060E9DD735@moltingpenguin.com> <5002AAAB.6080702@Media-Brokers.com> <1A94AD81-8D4E-4110-BA64-339E0FCEFBAE@moultingpenguin.com> <5ABB2A9D-B94E-4244-8D84-13851D3E56EB@moltingpenguin.com> <5003F09F.40908@Media-Brokers.com> <50053C0A.5090208@Media-Brokers.com> <592BA186-388E-442B-86B8-4A1B36B9BBE6@moultingpenguin.com> <7319F6C7-948B-40F6-ACE4-3F8D87CA9072@moltingpenguin.com> Message-ID: On Jul 18, 2012, at 12:51 AM, Sean Kamath wrote: > Since dovecot-lda is passed the -d option, it's using that (and only that) for delivery. Since there's no X-Original-To: header, it doesn't really have any option (since the non-envelope-to address might not exist, or point to something other than the end user's mailbox, etc). > > It would appear that I have two options: > > 1) Figure out what macro(s) is(are) valid to pass to dovecot-lda with the -a option in the sendmail config file (sendmail.cf). > > 2) Get sendmail to put "X-Original-To:
" in the header. > > Both would appear to be a sendmail issue, not a dovecot/pigeonhole issue. OK, for the archives: I went with Option 2. In my sendmail.mc, I added: LOCAL_CONFIG H?${u}?X-Original-To: $u This adds the header X-Original-To with the envelope recipient as the value. Seems like a hack, but what can you do? It works. BTW: No one asked, but I thought I'd just mention that the reason I want to use the envelope recipient address is many bulk-mailing lists will use blind addressing, so you can't use the To: or Cc: headers effectively. Anyway, sorry for the noise everyone. Sean From alessio at skye.it Thu Jul 19 12:05:35 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 19 Jul 2012 11:05:35 +0200 Subject: [Dovecot] Make a whole mailbox read-only In-Reply-To: <4FFC502F.2090004@dada.eu> References: <4FFC502F.2090004@dada.eu> Message-ID: <5007CDDF.7030203@skye.it> Il 10/07/2012 17:54, Sandro Tosi ha scritto: > Hello, > i've found an old thread [1] where it was stated that a whole mailbox > can be made read-only only setting the according permissions on the > filesystem side. > > [1] > http://www.mailinglistarchive.com/html/dovecot at dovecot.org/2011-12/msg00300.html > > > i'd like to check if it's still the only option we have with dovecot > v2.0.18 . > > Regards, Hi, read-only via filesystem is not a good idea. We use, on Dovecot 2.1, ACL for create mailbox (Maildir/) as "Archive" and read-only acess via IMAP, could be a solutions also for you. Ciao -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From CMarcus at Media-Brokers.com Thu Jul 19 13:02:43 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Thu, 19 Jul 2012 06:02:43 -0400 Subject: [Dovecot] Converting from Maildir format ? In-Reply-To: <50072D01.1000905@5sg.eu> References: <50072D01.1000905@5sg.eu> Message-ID: <5007DB43.9060403@Media-Brokers.com> On 2012-07-18 5:39 PM, Nick at 5sg.eu wrote: > I'm still trying to find out if anyone knows the correct > way to either switch the existing Maildir multiple file format to > the single file format I believe known as MBox. http://wiki2.dovecot.org/Migration/MailFormat Fyi, Timo has designed a new higher performance format known as dbox, for which there are two versions, you can read about here: http://wiki2.dovecot.org/MailboxFormat/dbox But you only want to use it if you are on more recent versions (at least latest 2.0.x). > Failing that, though it'll mean a LOT of work for me to manually > convert all of the existing Maildirs, how do I get dovecot to > work in future in MBox single file format. http://wiki2.dovecot.org/MailLocation/mbox > I tried entering 'mail_location = mbox:~/mail:INBOX=/%h/%u' in > the 'dovecot.conf', but it still saves each message in a file of > its own in the /new /cur /tmp directories ?? The wiki/docs are pretty thorough for basic settings... -- Best regards, Charles From ewald.lists at fun.de Thu Jul 19 14:16:06 2012 From: ewald.lists at fun.de (Ewald Dieterich) Date: Thu, 19 Jul 2012 13:16:06 +0200 Subject: [Dovecot] Append with literal+ and mailbox over quota Message-ID: <5007EC76.8040408@fun.de> I'm appending a mail with literal+ to a mailbox that is over quota: 1 APPEND INBOX {127+} From: To: Subject: Quota-Test Message-ID: Quota-Test 1 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) 2 NOOP 1 BAD Error in IMAP command APPEND: Invalid arguments. 3 NOOP 3 OK NOOP completed. Somehow the first NOOP is considered to be part of the already failed APPEND. I have to add an empty line after to NO response for the NOOP to work: 1 APPEND INBOX {127+} From: To: Subject: Quota-Test Message-ID: Quota-Test 1 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) 2 NOOP 2 OK NOOP completed. Can somebody confirm this to be bug? Here is the output of doveconf -n # 2.1.8: /usr/local/dovecot-orig/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.3 ext3 auth_debug = yes auth_verbose = yes debug_log_path = /var/log/dovecot/debug.log disable_plaintext_auth = no info_log_path = /var/log/dovecot/info.log log_path = /var/log/dovecot/error.log mail_debug = yes mail_location = mdbox:/var/spool/dovecot/%2.256Hu/%3.4096Hu/%u mail_plugins = notify mail_log quota acl namespace { inbox = yes location = prefix = INBOX. separator = . type = private } passdb { driver = pam } plugin { acl = vfile autocreate = INBOX.Drafts autocreate2 = INBOX.Sent autocreate3 = INBOX.Trash quota = dict:User quota::file:/var/spool/dovecot/%2.256Hu/%3.4096/%u/dovecot.dict quota_rule = *:bytes=2000 } ssl = no userdb { driver = passwd } protocol imap { mail_plugins = notify mail_log quota acl autocreate imap_quota imap_acl From Ralf.Hildebrandt at charite.de Thu Jul 19 15:45:00 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 14:45:00 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? Message-ID: <20120719124500.GA24108@charite.de> Hi! Anybody got a doveadm script which can remove leading and trailing spaces from folder names? Right now we're migrating mailboxes from dovecot -> Exchange, and Exchange cannot handle leading and trailing spaces in Folder names. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From xchris89x at googlemail.com Thu Jul 19 15:56:57 2012 From: xchris89x at googlemail.com (Chris) Date: Thu, 19 Jul 2012 14:56:57 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719124500.GA24108@charite.de> References: <20120719124500.GA24108@charite.de> Message-ID: 2012/7/19 Ralf Hildebrandt : > Right now we're migrating mailboxes from dovecot -> Exchange, and > Exchange cannot handle leading and trailing spaces in Folder names. You're migrating to M$ exchange??? :o -- Chris From Ralf.Hildebrandt at charite.de Thu Jul 19 15:59:46 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 14:59:46 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> Message-ID: <20120719125946.GB24108@charite.de> * Chris : > 2012/7/19 Ralf Hildebrandt : > > > Right now we're migrating mailboxes from dovecot -> Exchange, and > > Exchange cannot handle leading and trailing spaces in Folder names. > > You're migrating to M$ exchange??? :o Don't ask. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From robert at schetterer.org Thu Jul 19 16:07:23 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Jul 2012 15:07:23 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719124500.GA24108@charite.de> References: <20120719124500.GA24108@charite.de> Message-ID: <5008068B.3030603@schetterer.org> Am 19.07.2012 14:45, schrieb Ralf Hildebrandt: > Hi! > > Anybody got a doveadm script which can remove leading and trailing > spaces from folder names? > > Right now we're migrating mailboxes from dovecot -> Exchange, and > Exchange cannot handle leading and trailing spaces in Folder names. > use imapsync ? http://imapsync.lamiral.info/imapsync_example.bat.txt REM imapsync example batch for Windows users REM lines beginning with REM are just comments REM Replace imap.foo.org toto secretoto 192.168.42.4 titi secretiti with your own values .\imapsync.exe --host1 imap.foo.org --user1 toto --password1 "secretoto" --host2 192.168.42.4 --user2 titi --password2 "secretiti" Buy standalone imapsync.exe for win32 Struggle free from source code and Perl installation by buying the latest win32 standalone imapsync.exe for 42 EUR + One year of imapsync updates without extra payment. + 30 days money-back guarantee! thats the real punish for going exchnage *g -- Best Regards MfG Robert Schetterer From Ralf.Hildebrandt at charite.de Thu Jul 19 16:16:31 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 15:16:31 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <5008068B.3030603@schetterer.org> References: <20120719124500.GA24108@charite.de> <5008068B.3030603@schetterer.org> Message-ID: <20120719131630.GC24108@charite.de> * Robert Schetterer : > Am 19.07.2012 14:45, schrieb Ralf Hildebrandt: > > Hi! > > > > Anybody got a doveadm script which can remove leading and trailing > > spaces from folder names? > > > > Right now we're migrating mailboxes from dovecot -> Exchange, and > > Exchange cannot handle leading and trailing spaces in Folder names. > > > > use imapsync ? No, that doesn't work, since Exchange doesn't accept the folder names. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Thu Jul 19 16:19:55 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 15:19:55 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719131630.GC24108@charite.de> References: <20120719124500.GA24108@charite.de> <5008068B.3030603@schetterer.org> <20120719131630.GC24108@charite.de> Message-ID: <20120719131954.GD24108@charite.de> * Ralf Hildebrandt : > > use imapsync ? > > No, that doesn't work, since Exchange doesn't accept the folder names. OK, in some cases one can use --regextrans2 "s/Sent$/Sent Items/g" \ --regextrans2 "s/Drafts$/Entw&APw-rfe/g" \ --regextrans2 "s/spam$/Junk-E-Mail/g" \ --regextrans2 "s/(.*) $/$1/g" \ but that cannot handle leading & trailing spaces in the foldernames -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Ralf.Hildebrandt at charite.de Thu Jul 19 16:24:55 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 15:24:55 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719131954.GD24108@charite.de> References: <20120719124500.GA24108@charite.de> <5008068B.3030603@schetterer.org> <20120719131630.GC24108@charite.de> <20120719131954.GD24108@charite.de> Message-ID: <20120719132454.GE24108@charite.de> * Ralf Hildebrandt : > * Ralf Hildebrandt : > > > > use imapsync ? > > > > No, that doesn't work, since Exchange doesn't accept the folder names. > > OK, in some cases one can use > > --regextrans2 "s/Sent$/Sent Items/g" \ > --regextrans2 "s/Drafts$/Entw&APw-rfe/g" \ > --regextrans2 "s/spam$/Junk-E-Mail/g" \ > --regextrans2 "s/(.*) $/$1/g" \ > > but that cannot handle leading & trailing spaces in the foldernames in intermediate foldernames... It can fix: "INBOX.foo " but can't fix "INBOX. foo. bar" -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From robert at schetterer.org Thu Jul 19 16:31:18 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Jul 2012 15:31:18 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719132454.GE24108@charite.de> References: <20120719124500.GA24108@charite.de> <5008068B.3030603@schetterer.org> <20120719131630.GC24108@charite.de> <20120719131954.GD24108@charite.de> <20120719132454.GE24108@charite.de> Message-ID: <50080C26.7070107@schetterer.org> Am 19.07.2012 15:24, schrieb Ralf Hildebrandt: > * Ralf Hildebrandt : >> * Ralf Hildebrandt : >> >>>> use imapsync ? >>> >>> No, that doesn't work, since Exchange doesn't accept the folder names. >> >> OK, in some cases one can use >> >> --regextrans2 "s/Sent$/Sent Items/g" \ >> --regextrans2 "s/Drafts$/Entw&APw-rfe/g" \ >> --regextrans2 "s/spam$/Junk-E-Mail/g" \ >> --regextrans2 "s/(.*) $/$1/g" \ >> >> but that cannot handle leading & trailing spaces in the foldernames > > in intermediate foldernames... It can fix: > > "INBOX.foo " > but can't fix > "INBOX. foo. bar" > thats a beginn, i looking to a solution too -- Best Regards MfG Robert Schetterer From robert at schetterer.org Thu Jul 19 16:47:53 2012 From: robert at schetterer.org (Robert Schetterer) Date: Thu, 19 Jul 2012 15:47:53 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50080C26.7070107@schetterer.org> References: <20120719124500.GA24108@charite.de> <5008068B.3030603@schetterer.org> <20120719131630.GC24108@charite.de> <20120719131954.GD24108@charite.de> <20120719132454.GE24108@charite.de> <50080C26.7070107@schetterer.org> Message-ID: <50081009.4030209@schetterer.org> Am 19.07.2012 15:31, schrieb Robert Schetterer: > Am 19.07.2012 15:24, schrieb Ralf Hildebrandt: >> * Ralf Hildebrandt : >>> * Ralf Hildebrandt : >>> >>>>> use imapsync ? >>>> >>>> No, that doesn't work, since Exchange doesn't accept the folder names. >>> >>> OK, in some cases one can use >>> >>> --regextrans2 "s/Sent$/Sent Items/g" \ >>> --regextrans2 "s/Drafts$/Entw&APw-rfe/g" \ >>> --regextrans2 "s/spam$/Junk-E-Mail/g" \ >>> --regextrans2 "s/(.*) $/$1/g" \ >>> >>> but that cannot handle leading & trailing spaces in the foldernames >> >> in intermediate foldernames... It can fix: >> >> "INBOX.foo " >> but can't fix >> "INBOX. foo. bar" >> > > thats a beginn, i looking to a solution too > there is a tool for this no idea if it really working and fits to the problem http://www.microsoft.com/en-us/download/details.aspx?id=1329 Overview Transporter Suite contains a set of tools for migrations from both Lotus Domino Servers, and generic IMAP/POP sources. For Lotus Domino the suite contains a set of tools for Directory and Free/Busy interoperability between Lotus Domino 6, 7 and 8 and Exchange Server 2007 and Windows Server 2003/2008 Active Directory. In addition for Lotus Domino the suite contains migration tools to migrate users, groups, personal address lists, mailboxes, personal mail archives, and applications from Lotus Domino 5, 6, 7 or 8 to Active Directory, Exchange Server 2007, and Windows SharePoint Services 3.0. --here it comes For generic POP/IMAP servers the suite contains a set of tools to bulk migrate mailboxes from any generic email servers that support the POP3 or IMAP4 protocol to mailboxes in Exchange Server 2007 ------------- as ever there are problems with it http://social.technet.microsoft.com/forums/en-US/exchangesvrmigration/thread/b2111b89-beb9-4e63-b301-8eaf14447b09/ http://social.technet.microsoft.com/Forums/en-US/exchangesvrmigration/thread/319648ac-c248-400f-ba64-bb882018c790/#aa2b1c26-88a3-496d-8a8e-7a2dc4e0ebee https://msmvps.com/blogs/andersonpatricio/archive/2008/02/04/microsoft-transporter-suite-for-internet-mailboxes-configuring-exchange-impersonate-permission-in-exchange-server-2007.aspx some other tool IMAP to Exchange 2003 - Exchange Migration Wizard which runs at batch mode too http://technet.microsoft.com/en-us/library/aa998102.aspx http://technet.microsoft.com/en-us/library/bb123854.aspx if you need contact to exchange specialist i can give you some advanced one i remember working on pup folders with pfadmin, this tools semms now to have more options, perhaps help http://www.msexchange.org/articles/PFDavAdmin-tool-Part2.html -- Best Regards MfG Robert Schetterer From lists at wildgooses.com Thu Jul 19 16:52:38 2012 From: lists at wildgooses.com (Ed W) Date: Thu, 19 Jul 2012 14:52:38 +0100 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719124500.GA24108@charite.de> References: <20120719124500.GA24108@charite.de> Message-ID: <50081126.9000707@wildgooses.com> On 19/07/2012 13:45, Ralf Hildebrandt wrote: > Hi! > > Anybody got a doveadm script which can remove leading and trailing > spaces from folder names? > > Right now we're migrating mailboxes from dovecot -> Exchange, and > Exchange cannot handle leading and trailing spaces in Folder names. > Caveat that it might cause some wierd symptoms with clients, why not attack the dovecot mail backend and rename folders + sed the subscription files? Something like "find | rename" Good luck Ed W From Ralf.Hildebrandt at charite.de Thu Jul 19 17:07:38 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 16:07:38 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50081126.9000707@wildgooses.com> References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> Message-ID: <20120719140738.GI24108@charite.de> * Ed W : > On 19/07/2012 13:45, Ralf Hildebrandt wrote: > >Hi! > > > >Anybody got a doveadm script which can remove leading and trailing > >spaces from folder names? > > > >Right now we're migrating mailboxes from dovecot -> Exchange, and > >Exchange cannot handle leading and trailing spaces in Folder names. > > > > Caveat that it might cause some wierd symptoms with clients, why not > attack the dovecot mail backend and rename folders + sed the > subscription files? That's what I asked for: "Anybody got a doveadm script which can remove leading and trailing spaces from folder names?" -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From Daniel.Pieper at mentana.de Thu Jul 19 17:28:57 2012 From: Daniel.Pieper at mentana.de (Daniel Pieper) Date: Thu, 19 Jul 2012 14:28:57 +0000 Subject: [Dovecot] Disable W= in filenames Message-ID: <5E24DAB287DDB44CA15AE0C4C7D2B5D0107DD6E5@MentDCEX01.red007.local> Hi, I'm using Dovecot 2.1.8. Is it possible to deactivate the "W=" Maildir filename extension, so that dovecot not add it to maildir filenames? From alessio at skye.it Thu Jul 19 18:06:56 2012 From: alessio at skye.it (Alessio Cecchi) Date: Thu, 19 Jul 2012 17:06:56 +0200 Subject: [Dovecot] Disable W= in filenames In-Reply-To: <5E24DAB287DDB44CA15AE0C4C7D2B5D0107DD6E5@MentDCEX01.red007.local> References: <5E24DAB287DDB44CA15AE0C4C7D2B5D0107DD6E5@MentDCEX01.red007.local> Message-ID: <50082290.7030302@skye.it> Il 19/07/2012 16:28, Daniel Pieper ha scritto: > Hi, > I'm using Dovecot 2.1.8. Is it possible to deactivate the "W=" Maildir filename extension, so that dovecot not add it to maildir filenames? > Why out want disable it? -- Alessio Cecchi is: @ ILS -> http://www.linux.it/~alessice/ on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi GNU/Linux -> http://www.cecchi.biz/ @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it From ewald.lists at fun.de Thu Jul 19 18:21:18 2012 From: ewald.lists at fun.de (Ewald Dieterich) Date: Thu, 19 Jul 2012 17:21:18 +0200 Subject: [Dovecot] Append with literal+ and mailbox over quota In-Reply-To: <5007EC76.8040408@fun.de> References: <5007EC76.8040408@fun.de> Message-ID: <500825EE.9080104@fun.de> On 07/19/12 13:16, Ewald Dieterich wrote: > I'm appending a mail with literal+ to a mailbox that is over quota: > > 1 APPEND INBOX {127+} [...] > 1 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) > 2 NOOP > 1 BAD Error in IMAP command APPEND: Invalid arguments. > 3 NOOP > 3 OK NOOP completed. > > Somehow the first NOOP is considered to be part of the already failed > APPEND. I have to add an empty line after to NO response for the NOOP to > work: > > 1 APPEND INBOX {127+} [...] > 1 NO [OVERQUOTA] Quota exceeded (mailbox for user is full) > > 2 NOOP > 2 OK NOOP completed. > > Can somebody confirm this to be bug? I think I'm just confused because Dovecot sends the NO reply before I finish the string literal. In the first case I send an invalid string literal (which Dovecot tries to interpret as a MULTIAPPEND) and in the second case I finish the string literal correctly with CRLF. So I guess everything is OK. From hsn at filez.com Thu Jul 19 19:02:42 2012 From: hsn at filez.com (Radim Kolar) Date: Thu, 19 Jul 2012 18:02:42 +0200 Subject: [Dovecot] improve error messages with quotes Message-ID: <50082FA2.5050106@filez.com> I propose to improve error messages but inserting quotes. I spent good time until I looked into source code to find right meaning of error message. I would like them in following format: namespace configuration error no 'hidden=no' namespaces From Daniel.Pieper at mentana.de Thu Jul 19 21:18:49 2012 From: Daniel.Pieper at mentana.de (Daniel Pieper) Date: Thu, 19 Jul 2012 18:18:49 +0000 Subject: [Dovecot] Disable W= in filenames Message-ID: <5E24DAB287DDB44CA15AE0C4C7D2B5D0107DDBDB@MentDCEX01.red007.local> I implemented a plugin for dovecot and now i got a error that maildir filename has wrong W value. When i rename the filename manually and delete the W value, all is ok. From wojtek at wojtek.tensor.gdynia.pl Thu Jul 19 23:13:54 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 19 Jul 2012 22:13:54 +0200 (CEST) Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719124500.GA24108@charite.de> References: <20120719124500.GA24108@charite.de> Message-ID: > > Right now we're migrating mailboxes from dovecot -> Exchange, and > Exchange cannot handle leading and trailing spaces in Folder names. ask microsoft From h.reindl at thelounge.net Thu Jul 19 23:19:05 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 19 Jul 2012 22:19:05 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> Message-ID: <50086BB9.4040203@thelounge.net> Am 19.07.2012 22:13, schrieb Wojciech Puchar: >> >> Right now we're migrating mailboxes from dovecot -> Exchange, and >> Exchange cannot handle leading and trailing spaces in Folder names. > > ask microsoft another example what low-brained guy you are do you recognize that you even stripped the question? do you really think that he migrates to exchange because he have fun with it? -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From wojtek at wojtek.tensor.gdynia.pl Thu Jul 19 23:22:05 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Thu, 19 Jul 2012 22:22:05 +0200 (CEST) Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50086BB9.4040203@thelounge.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: >>> Right now we're migrating mailboxes from dovecot -> Exchange, and >>> Exchange cannot handle leading and trailing spaces in Folder names. >> >> ask microsoft > > another example what low-brained guy you are You are scared. From Ralf.Hildebrandt at charite.de Fri Jul 20 00:27:31 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Thu, 19 Jul 2012 23:27:31 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50086BB9.4040203@thelounge.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: <20120719212731.GH18426@charite.de> * Reindl Harald : > do you really think that he migrates to exchange > because he have fun with it? It seems to be a well known restriction, and folder names with leading or trailing spaces do indeed make little sense. In all cases those that I found were typos. From jerry at seibercom.net Fri Jul 20 01:13:03 2012 From: jerry at seibercom.net (Jerry) Date: Thu, 19 Jul 2012 18:13:03 -0400 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719212731.GH18426@charite.de> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <20120719212731.GH18426@charite.de> Message-ID: <20120719181303.105dbab3@scorpio> On Thu, 19 Jul 2012 23:27:31 +0200 Ralf Hildebrandt articulated: > * Reindl Harald : > > > do you really think that he migrates to exchange > > because he have fun with it? > > It seems to be a well known restriction, and folder names with > leading or trailing spaces do indeed make little sense. In all cases > those that I found were typos. I posted on the MS TechNet site an was told that there doesn't appear to be any support for it. Does anyone know of an actual RFC that specifically deals with this? I could not find one, but it was just a quick perusal. Personally, I agree with Reindl. Leading spaces don't seem to make any sense at all and I have not figured out why I would want a trailing one either. -- Jerry ? "There are seldom good technological solutions to behavioral problems." From robert at schetterer.org Fri Jul 20 07:58:38 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 20 Jul 2012 06:58:38 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719181303.105dbab3@scorpio> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <20120719212731.GH18426@charite.de> <20120719181303.105dbab3@scorpio> Message-ID: <5008E57E.7070703@schetterer.org> Am 20.07.2012 00:13, schrieb Jerry: > On Thu, 19 Jul 2012 23:27:31 +0200 > Ralf Hildebrandt articulated: > >> * Reindl Harald : >> >>> do you really think that he migrates to exchange >>> because he have fun with it? >> >> It seems to be a well known restriction, and folder names with >> leading or trailing spaces do indeed make little sense. In all cases >> those that I found were typos. > > I posted on the MS TechNet site an was told that there doesn't appear > to be any support for it. Does anyone know of an actual RFC that > specifically deals with this? I could not find one, but it was just a > quick perusal. Personally, I agree with Reindl. Leading spaces don't > seem to make any sense at all and I have not figured out why I would > want a trailing one either. > > i found that such folders work in exchange might work in version 2000/2003 and do not in 2007, so it seems that must be worked out in a pure exchange upgrade too, so migration ms transporter suite should handle it somehow, however Ralfs Problem seems to be so rare that it could more easy solved rename that folder by hand so we should now get back to dovecot support *g -- Best Regards MfG Robert Schetterer From Nick at 5sg.eu Fri Jul 20 08:13:06 2012 From: Nick at 5sg.eu (Nick at 5sg.eu) Date: Fri, 20 Jul 2012 01:13:06 -0400 Subject: [Dovecot] Converting from Maildir format ? Message-ID: <5008E8E2.3060806@5sg.eu> Hi Stan, >> I'm still trying to find out if anyone knows the correct >> way to either switch the existing Maildir multiple file >> format to the single file format I believe known as MBox. > Did you restart dovecot after the mailbox_location change? Yup... > Show 'dovecot -n' output It shows it - and 'plugin: convert_mail: mbox:%h/%u/mail' I also changed it in dovecot/conf.d/01-dovecot-postfix.conf Hi Charles, > format known as dbox Running v1.2.9 on Ubuntu v10.4 > The wiki/docs are pretty thorough for basic settings. That's where I got the mbox line above.. Thanks gentlemen, Nick. . . . . -- _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ . . . . . . . . ...---... (Stamp Out Spam and Save Our Servers) http://sosNow.ws -=[ do you know morse code ? ]=- . . . . . . . . _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ From wojtek at wojtek.tensor.gdynia.pl Fri Jul 20 10:48:30 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 20 Jul 2012 09:48:30 +0200 (CEST) Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50086BB9.4040203@thelounge.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: > do you really think that he migrates to exchange > because he have fun with it? so? is it my problem that he works in company managed by an idiot? Why dovecot users have to provide free support for microsoft software, while microsoft are paid for their "products" thousands of dollars? Does it make sense? From f.bonnet at esiee.fr Fri Jul 20 10:56:52 2012 From: f.bonnet at esiee.fr (Frank Bonnet) Date: Fri, 20 Jul 2012 09:56:52 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: <50090F44.7000807@esiee.fr> On 07/20/2012 09:48 AM, Wojciech Puchar wrote: >> do you really think that he migrates to exchange >> because he have fun with it? > so? is it my problem that he works in company managed by an idiot? > > Why dovecot users have to provide free support for microsoft software, > while microsoft are paid for their "products" thousands of dollars? > > Does it make sense? +1 From tom at whyscream.net Fri Jul 20 11:05:05 2012 From: tom at whyscream.net (Tom Hendrikx) Date: Fri, 20 Jul 2012 10:05:05 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50090F44.7000807@esiee.fr> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <50090F44.7000807@esiee.fr> Message-ID: <50091131.2000700@whyscream.net> On 7/20/12 9:56 AM, Frank Bonnet wrote: > On 07/20/2012 09:48 AM, Wojciech Puchar wrote: >>> do you really think that he migrates to exchange >>> because he have fun with it? >> so? is it my problem that he works in company managed by an idiot? >> >> Why dovecot users have to provide free support for microsoft software, >> while microsoft are paid for their "products" thousands of dollars? >> >> Does it make sense? > > +1 > -1 A dovecot specific question was asked by the OP, but people seem to think this is a reason to bash either the OP, his company or non-dovecot software (exchange) for something which is related to the question, but off-topic for the list. If you don't have any dovecot related input for the OP, just leave the thread alone, in stead of trolling. Thank you very much. -- Tom From role.Dovecot-Readers at JLAssocs.com Fri Jul 20 11:13:37 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Fri, 20 Jul 2012 09:13:37 +0100 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50091131.2000700@whyscream.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <50090F44.7000807@esiee.fr> <50091131.2000700@whyscream.net> Message-ID: On 20 Jul 2012, at 09:05, Tom Hendrikx wrote: > On 7/20/12 9:56 AM, Frank Bonnet wrote: >> On 07/20/2012 09:48 AM, Wojciech Puchar wrote: >>>> do you really think that he migrates to exchange >>>> because he have fun with it? >>> so? is it my problem that he works in company managed by an idiot? >>> >>> Why dovecot users have to provide free support for microsoft software, >>> while microsoft are paid for their "products" thousands of dollars? >>> >>> Does it make sense? >> >> +1 >> > > -1 > > A dovecot specific question was asked by the OP, but people seem to > think this is a reason to bash either the OP, his company or non-dovecot > software (exchange) for something which is related to the question, but > off-topic for the list. > > If you don't have any dovecot related input for the OP, just leave the > thread alone, in stead of trolling. Thank you very much. +1e100 From h.reindl at thelounge.net Fri Jul 20 11:23:38 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 20 Jul 2012 10:23:38 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: <5009158A.9010106@thelounge.net> Am 20.07.2012 09:48, schrieb Wojciech Puchar: >> do you really think that he migrates to exchange >> because he have fun with it? > so? is it my problem that he works in company managed by an idiot? so why do you not simply shut ip at all? YOU are really not in the position to qualify anybody as idiot, everybody but you can do this > Why dovecot users have to provide free support for microsoft > software, while microsoft are paid for their > "products" thousands of dollars? his question was if someone has a ready script to rename the folders - so hwat is your problem? > Does it make sense? yes it makes sense that someone requests a possible existing ready script instead start to write one, but as said you foolish idiot even stripped the real question of the OP before your arrogant answer -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From wojtek at wojtek.tensor.gdynia.pl Fri Jul 20 11:30:52 2012 From: wojtek at wojtek.tensor.gdynia.pl (Wojciech Puchar) Date: Fri, 20 Jul 2012 10:30:52 +0200 (CEST) Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <5009158A.9010106@thelounge.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <5009158A.9010106@thelounge.net> Message-ID: >> so? is it my problem that he works in company managed by an idiot? > > so why do you not simply shut ip at all? You are even more scared. From h.reindl at thelounge.net Fri Jul 20 11:36:38 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 20 Jul 2012 10:36:38 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <5009158A.9010106@thelounge.net> Message-ID: <50091896.60408@thelounge.net> Am 20.07.2012 10:30, schrieb Wojciech Puchar: >>> so? is it my problem that he works in company managed by an idiot? >> >> so why do you not simply shut ip at all? > You are even more scared i am not scared i am only wondering why you are prove permanently that you are too stupid to understand basics like that there is no need for "reply all" on a mailing list, but thank god my dbmail backend supports "suppress_duplicates = yes" after the dovecot-proxy in front....... really, after your off-list explainations why you are the only one who can build up storage systems while SAN-storage vendors are unable to do this you can not scare me any more -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From robert at schetterer.org Fri Jul 20 11:45:54 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 20 Jul 2012 10:45:54 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: <50091AC2.6090205@schetterer.org> Am 20.07.2012 09:48, schrieb Wojciech Puchar: >> do you really think that he migrates to exchange >> because he have fun with it? > so? is it my problem that he works in company managed by an idiot? > > Why dovecot users have to provide free support for microsoft software, > while microsoft are paid for their "products" thousands of dollars? > > Does it make sense? Hi Wojciech, this is a free world, so if anyone want to give help why he shouldnt, of course its a little off topic in the dovecot list talking to exchange, but at last ,it doesnt hurt anyone, as long it gets no flood theme, anyway ,small knowings other servers is always fine, we dont live in a dovecot postfix world and gnu is no substitute for religion, the main goal of every tec mail list is to solve tec problems. At last ,i think pure M$ lovers arent booking this list that much that you need to be afraid of them *g -- Best Regards MfG Robert Schetterer From xchris89x at googlemail.com Fri Jul 20 12:10:44 2012 From: xchris89x at googlemail.com (Chris) Date: Fri, 20 Jul 2012 11:10:44 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: 2012/7/20 Wojciech Puchar : >> do you really think that he migrates to exchange >> because he have fun with it? > > so? is it my problem that he works in company managed by an idiot? > > Why dovecot users have to provide free support for microsoft software, while > microsoft are paid for their "products" thousands of dollars? > > Does it make sense? Do you think Ralf is migrating just for fun to M$ exchange? Ralf is certainly no friend of M$ software. I think he's freaked out when he heard about it. -- Chris From CMarcus at Media-Brokers.com Fri Jul 20 12:24:24 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Jul 2012 05:24:24 -0400 Subject: [Dovecot] Converting from Maildir format ? In-Reply-To: <5008E8E2.3060806@5sg.eu> References: <5008E8E2.3060806@5sg.eu> Message-ID: <500923C8.1010700@Media-Brokers.com> On 2012-07-20 1:13 AM, Nick at 5sg.eu wrote: >> Show 'dovecot -n' output > It shows it - and 'plugin: convert_mail: mbox:%h/%u/mail' > I also changed it in dovecot/conf.d/01-dovecot-postfix.conf That is not helpful... when asked for dovecot -n output, please post the FULL output, not some copy/paste plus your evaluation of what you see. -- Best regards, Charles From CMarcus at Media-Brokers.com Fri Jul 20 12:27:03 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Fri, 20 Jul 2012 05:27:03 -0400 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <5009158A.9010106@thelounge.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <5009158A.9010106@thelounge.net> Message-ID: <50092467.3040208@Media-Brokers.com> On 2012-07-20 4:23 AM, Reindl Harald wrote: > Am 20.07.2012 09:48, schrieb Wojciech Puchar: >>> do you really think that he migrates to exchange because he have >>> fun with it? >> so? is it my problem that he works in company managed by an idiot? > so why do you not simply shut ip at all? Reindl... PLEASE stop replying to this troll, you just make the problem worse - you're better off sending him to /dev/null like I (and many others) have already... but when *you* reply, you defeat my filters. -- Best regards, Charles From jerry at seibercom.net Fri Jul 20 13:22:37 2012 From: jerry at seibercom.net (Jerry) Date: Fri, 20 Jul 2012 06:22:37 -0400 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50091131.2000700@whyscream.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <50090F44.7000807@esiee.fr> <50091131.2000700@whyscream.net> Message-ID: <20120720062237.42b1e9e0@scorpio> On Fri, 20 Jul 2012 10:05:05 +0200 Tom Hendrikx articulated: > On 7/20/12 9:56 AM, Frank Bonnet wrote: > > On 07/20/2012 09:48 AM, Wojciech Puchar wrote: > >>> do you really think that he migrates to exchange > >>> because he have fun with it? > >> so? is it my problem that he works in company managed by an idiot? > >> > >> Why dovecot users have to provide free support for microsoft > >> software, while microsoft are paid for their "products" thousands > >> of dollars? > >> > >> Does it make sense? > > > > +1 > > -1 > > A dovecot specific question was asked by the OP, but people seem to > think this is a reason to bash either the OP, his company or > non-dovecot software (exchange) for something which is related to the > question, but off-topic for the list. > > If you don't have any dovecot related input for the OP, just leave the > thread alone, in stead of trolling. Thank you very much. +1 -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From jerry at seibercom.net Fri Jul 20 13:30:31 2012 From: jerry at seibercom.net (Jerry) Date: Fri, 20 Jul 2012 06:30:31 -0400 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50092467.3040208@Media-Brokers.com> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> <5009158A.9010106@thelounge.net> <50092467.3040208@Media-Brokers.com> Message-ID: <20120720063031.1ad208a6@scorpio> On Fri, 20 Jul 2012 05:27:03 -0400 Charles Marcus articulated: > On 2012-07-20 4:23 AM, Reindl Harald wrote: > > Am 20.07.2012 09:48, schrieb Wojciech Puchar: > >>> do you really think that he migrates to exchange because he have > >>> fun with it? > > >> so? is it my problem that he works in company managed by an idiot? > > > so why do you not simply shut ip at all? > > Reindl... PLEASE stop replying to this troll, you just make the > problem worse - you're better off sending him to /dev/null like I > (and many others) have already... but when *you* reply, you defeat my > filters. Charles, I totally agree. This TROLL should just be banned by Timo. I wish he was banned from the FreeBSD* list(s) also. Apparently he is nothing more than a small SOHO employee with no factual knowledge or anything of value to contribute. I have filters in place to bounce his mail; however, when someone replies to his garbage, as you stated, it defeats the effectiveness of TCM (TROLL Counter Measures). -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ From jerry at seibercom.net Fri Jul 20 14:22:59 2012 From: jerry at seibercom.net (Jerry) Date: Fri, 20 Jul 2012 07:22:59 -0400 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <50086BB9.4040203@thelounge.net> References: <20120719124500.GA24108@charite.de> <50086BB9.4040203@thelounge.net> Message-ID: <20120720072259.30a104c7@scorpio> On Thu, 19 Jul 2012 22:19:05 +0200 Reindl Harald articulated: > > > Am 19.07.2012 22:13, schrieb Wojciech Puchar: > >> > >> Right now we're migrating mailboxes from dovecot -> Exchange, and > >> Exchange cannot handle leading and trailing spaces in Folder names. > > > > ask microsoft > > another example what low-brained guy you are > do you recognize that you even stripped the question? > > do you really think that he migrates to exchange > because he have fun with it? Reindl, this reminds me of an old quote that is very pertinent to Wojciech: "Never argue with a fool - they will drag you down to their level, then beat you with experience." -- Jerry ? Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __________________________________________________________________ -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 488 bytes Desc: not available URL: From stan at hardwarefreak.com Fri Jul 20 15:23:07 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Fri, 20 Jul 2012 07:23:07 -0500 Subject: [Dovecot] Converting from Maildir format ? In-Reply-To: <5008E8E2.3060806@5sg.eu> References: <5008E8E2.3060806@5sg.eu> Message-ID: <50094DAB.6030409@hardwarefreak.com> On 7/20/2012 12:13 AM, Nick at 5sg.eu wrote: > > Show 'dovecot -n' output > > It shows it - and 'plugin: convert_mail: mbox:%h/%u/mail' > I also changed it in dovecot/conf.d/01-dovecot-postfix.conf "Show 'dovecot -n' output" means exactly that. You did not. It's very difficult to help you if you refuse to provide the information requested. -- Stan From Juergen.Obermann at hrz.uni-giessen.de Fri Jul 20 15:46:04 2012 From: Juergen.Obermann at hrz.uni-giessen.de (=?iso-8859-1?b?SvxyZ2Vu?= Obermann) Date: Fri, 20 Jul 2012 14:46:04 +0200 Subject: [Dovecot] mdbox with separate index location results in many emptydirectories Message-ID: <20120720144604.12933ye5jvozyeyo@webmail.hrz.uni-giessen.de> Hallo, I use dovecot 2.1.7 with mdbox format and I specified a separate location for mailbox index files with mail_location = mdbox:/dovecot/storage%h:INDEX=/dovecot/index%h With this setting I get for every mailbox a directory under both filesystems /dovecot/index and /dovecot/storage, e.g. for the inbox it looks like that: # ls -lR /dovecot/index/home/hrz/myuser/mailboxes/INBOX /dovecot/storage/home/hrz/myuser/mailboxes/INBOX /dovecot/index/home/hrz/myuser/mailboxes/INBOX: total 49 -rw------- 1 myuser hrz 20480 Jul 20 11:12 dovecot.index.cache -rw------- 1 myuser hrz 1280 Jul 20 08:00 dovecot.index.log -rw------- 1 myuser hrz 301 Jul 19 16:45 dovecot.index.search -rw------- 1 myuser hrz 20 Jul 19 16:45 dovecot.index.search.uids /dovecot/storage/home/hrz/myuser/mailboxes/INBOX: total 3 drwx------ 2 myuser hrz 2 Jul 19 15:20 dbox-Mails /dovecot/storage/home/hrz/myuser/mailboxes/INBOX/dbox-Mails: total 0 As expected the index files now reside under /dovecot/index%h/mailboxes/, but the path /dovecot/storage%h/mailboxes/ also contains for every user and every mailbox name a directory which in turn contains only an empty subdirectory 'dbox-Mails'. These directories seem to me to sum up to a lot of unneded infrstructure objects. I tried to delete these empty directories, but dovecot did not like that and stopped to work. So somehow they are necessary, in spite of setting a separate index file location. So my question is, could these empty directories be avoided, perhaps in a future version? Greetings, Juergen -- Hochschulrechenzentrum der | Mail: Juergen.Obermann at hrz.uni-giessen.de Justus-Liebig-Universitaet | WWW: http://www.uni-giessen.de/obermann/ Heinrich-Buff-Ring 44 | Tel: 0641-99-13054 (0641-99-13001) D-35392 Giessen, Germany | Fax: 0641-99-13009 From gary at partis.co.uk Fri Jul 20 18:31:56 2012 From: gary at partis.co.uk (Gary Partis) Date: Fri, 20 Jul 2012 16:31:56 +0100 Subject: [Dovecot] postfix/sendmail configuration Message-ID: <9f971104c7b40c95baa57de219d1a336@partis.homeip.net> Hi all I have searched the net and cant find anything relating to this. In 15-lda.conf, the default sendmail is used to send messges from Dovecot, such as "out of hours" and "redirected" emails controlled by sieves. However, postfix/sendmail is looking in /etc/postfix/main.cf for the configuration file (as logged in /var/log/maillog), but I have the config file elsewhere, and I even have MAIL_CONFIG setup to point to the true config directory. Is there a method of setting 15-lda.conf sendmail config to ensure it uese the correct postfix config file? Thanks in advance Gary From andrzej.filip at gmail.com Fri Jul 20 22:32:20 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Fri, 20 Jul 2012 21:32:20 +0200 Subject: [Dovecot] postfix/sendmail configuration In-Reply-To: <9f971104c7b40c95baa57de219d1a336@partis.homeip.net> References: <9f971104c7b40c95baa57de219d1a336@partis.homeip.net> Message-ID: <5009B244.8020303@gmail.com> On 07/20/2012 05:31 PM, Gary Partis wrote: > I have searched the net and cant find anything relating to this. > > In 15-lda.conf, the default sendmail is used to send messges from > Dovecot, such as "out of hours" and "redirected" emails controlled by > sieves. > > However, postfix/sendmail is looking in /etc/postfix/main.cf for the > configuration file (as logged in /var/log/maillog), but I have the > config file elsewhere, and I even have MAIL_CONFIG setup to point to > the true config directory. > > Is there a method of setting 15-lda.conf sendmail config to ensure it > uese the correct postfix config file? Have you considered setting "sendmail_path" to your custom "sendmail wrapper" script? It should allow you to pass extra command line options to sendmail/postfix [e.g. non standard location of the configuration file]. IMHO on long run docvecot configuration should support passing extra command line parameters to sendmail (e.g. via "sendmail_options"). From hans.j.albertsson at branneriet.se Sat Jul 21 12:24:09 2012 From: hans.j.albertsson at branneriet.se (Hans J. Albertsson) Date: Sat, 21 Jul 2012 11:24:09 +0200 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver Message-ID: <500A7539.7060403@branneriet.se> I'm trying to set up a first test for a dovecot + postfix (+ many other things) mail server for about 1000 accounts, and I keep stumbling on confusing inconsistencies in the HowTo documents I'm trying to follow. I tried to do a simple setup with virtual users using a howto saying it was for just that and that it's for Dovecot 2.x, and it shows a few text chunks to put in files, and then says it's better to do the changes in the example-config conf file.... And then the whole structure of config files and dirs is totally different from what is implied in the howto. And of course it won't run. Probably because of the structure of a conf.d dir is multiple files, and the instructions are explicitly written for the case of having a single conf file. Can anyone suggest an order in which to read docs that will take me thru w/o being totally confusing? I would very much like to be able to enable bit by bit, rather than have to understand all of dovecot before I can even begin to test it. P.S: I want to setup a server using postfix+dovecot+tls+virtualusers+maildir+dspam+mailman, at least that's the target I'm aiming for. Please suggest a good HowTo for this setup that will not be too confusing. Also, Sorry if I sound like a whining kid: I am new to dovecot and postfix, but I was a software support engineer for Sun for ages, and I should really be able to manage this if I just had some consistent and non-confusing HowTo or Tutorial to follow as a first try. From pw at wk-serv.de Sat Jul 21 13:28:21 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 21 Jul 2012 12:28:21 +0200 Subject: [Dovecot] Director for smtp-auth Message-ID: <500A8445.4060601@wk-serv.de> Hi all, isn't it possible to use the director functionality for smtp-auth too? I have a frontend system for imap and outgoing mails using sasl and I recognized that all sasl username/password combinations are accepted as these credentials aren't checked by the backend servers. Regards Patrick From CMarcus at Media-Brokers.com Sat Jul 21 13:35:03 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sat, 21 Jul 2012 06:35:03 -0400 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver In-Reply-To: <500A7539.7060403@branneriet.se> References: <500A7539.7060403@branneriet.se> Message-ID: <500A85D7.7030900@Media-Brokers.com> On 2012-07-21 5:24 AM, Hans J. Albertsson wrote: > Also, Sorry if I sound like a whining kid: I am new to dovecot and > postfix, but I was a software support engineer for Sun for ages, and I > should really be able to manage this if I just had some consistent and > non-confusing HowTo or Tutorial to follow as a first try. 1. Which 'HowTo' were you following? 2. If it wasn't the dovecot 'official' docs on the wiki, try those: http://wiki2.dovecot.org/ $random_howtos on the internet is not the way to build a secure mail server. Some of the wiki2 docs still have some v1 cruft in them, but for the most part they should be all you need (along with asking questions here for clarification), and it is a wiki - anything that you encounter that you find confusing or incomplete, by all means come here for clarification, and then update the wiki to help others who encounter the same issues as you. You haven't even provided the most basic of details of the problem(s) you're having (like, for example, OS/platform, dovecot -n output, logs exhibiting the problem(s), etc... First, take a step back, and start with one thing at a time (testing and making sure each step is working as expected before moving to the next one)... 1. install postfix, secure it, and get it delivering mail to at least one (test if desired) virtual domain/users maildirs using its built-in virtual delivery agent, 2. add dovecot into the mix to serve up the users mail, 3. switch to the dovecot delivery agent (I recommend LMTP, but at a minimum use the Dovecot LDA), 4. enable dovecot-sasl, 5. integrate dspam, 6. integrate mailman Obviously, each step requires you to know how you want things to work - ie, do you want to provide POP+IMAP, or only IMAPS access, etc... Last - to answer the one specific complaint that you had - dovecot will use all of those separate config files in the conf.d directory, or you can put all of your settings into one config file, the choice is yours - see the wiki for details: http://wiki2.dovecot.org/BasicConfiguration -- Best regards, Charles From hans.j.albertsson at branneriet.se Sat Jul 21 14:07:27 2012 From: hans.j.albertsson at branneriet.se (Hans J. Albertsson) Date: Sat, 21 Jul 2012 13:07:27 +0200 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver In-Reply-To: <500A85D7.7030900@Media-Brokers.com> References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> Message-ID: <500A8D6F.1090808@branneriet.se> On 2012-07-21 12:35, Charles Marcus wrote: > On 2012-07-21 5:24 AM, Hans J. Albertsson > wrote: >> Also, Sorry if I sound like a whining kid: I am new to dovecot and >> postfix, but I was a software support engineer for Sun for ages, and I >> should really be able to manage this if I just had some consistent and >> non-confusing HowTo or Tutorial to follow as a first try. > > 1. Which 'HowTo' were you following? > > 2. If it wasn't the dovecot 'official' docs on the wiki, try those: > > http://wiki2.dovecot.org/ I followed the http://wiki2.dovecot.org/ -> HOWTOs, examples and tutorials -> _A simple virtual passwd file installation _ HowTo. > > $random_howtos on the internet is not the way to build a secure mail > server. > > Some of the wiki2 docs still have some v1 cruft in them, but for the > most part they should be all you need (along with asking questions > here for clarification), and it is a wiki - anything that you > encounter that you find confusing or incomplete, by all means come > here for clarification, and then update the wiki to help others who > encounter the same issues as you. > > You haven't even provided the most basic of details of the problem(s) > you're having (like, for example, OS/platform, dovecot -n output, logs > exhibiting the problem(s), etc... > > First, take a step back, and start with one thing at a time (testing > and making sure each step is working as expected before moving to the > next one)... > > 1. install postfix, secure it, and get it delivering mail to at least > one (test if desired) virtual domain/users maildirs using its built-in > virtual delivery agent, Done, tested, works. > > 2. add dovecot into the mix to serve up the users mail, And that's where I get lost: I keep fiddling back and forth wondering where I am... I suppose I should instead think like this: (comment, please!) Read the HowTo Look at the text chunks For each element, find where that happens in conf.d, and do it there, with the minimum change possible And then the LDA and later LMTP. I suppose if one succeeds in getting the LDA working, one can then move over to LMTP, with less effort? > > 3. switch to the dovecot delivery agent (I recommend LMTP, but at a > minimum use the Dovecot LDA), OK. > > 4. enable dovecot-sasl, > > 5. integrate dspam, > > 6. integrate mailman > > Obviously, each step requires you to know how you want things to work > - ie, do you want to provide POP+IMAP, or only IMAPS access, etc... > > Last - to answer the one specific complaint that you had - dovecot > will use all of those separate config files in the conf.d directory, > or you can put all of your settings into one config file, the choice > is yours - see the wiki for details: > > http://wiki2.dovecot.org/BasicConfiguration > From pw at wk-serv.de Sat Jul 21 15:02:05 2012 From: pw at wk-serv.de (Patrick Westenberg) Date: Sat, 21 Jul 2012 14:02:05 +0200 Subject: [Dovecot] Director for smtp-auth In-Reply-To: <500A8445.4060601@wk-serv.de> References: <500A8445.4060601@wk-serv.de> Message-ID: <500A9A3D.60500@wk-serv.de> > isn't it possible to use the director functionality for smtp-auth too? > > I have a frontend system for imap and outgoing mails using sasl and I > recognized that all sasl username/password combinations are accepted as > these credentials aren't checked by the backend servers. I found a workaround which works for me: http://dovecot.org/list/dovecot/2011-May/059153.html From hans.j.albertsson at branneriet.se Sat Jul 21 17:49:48 2012 From: hans.j.albertsson at branneriet.se (Hans J. Albertsson) Date: Sat, 21 Jul 2012 16:49:48 +0200 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver In-Reply-To: <500A8D6F.1090808@branneriet.se> References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> <500A8D6F.1090808@branneriet.se> Message-ID: <500AC18C.4030000@branneriet.se> Actually I now tried to follow slavishly one other HowTo from the dovecot.org "official (??? really??? ) web site: The VirtualUserFlatFilesPostfix HowTo... As an example of my woes: I hit a snag here when I got to the Mail Location and Namespaces bit. The config seems a bit confusing in that the public namespace config says location = maildir:/var/vmail/public:LAYOUT=fs:INDEX=~/public but the comments on the next line talks about User Home directory structure:/var/vmail/// I personally want the vmail user under /export/home/vmail, but that's not the confusion here. Isn't the part supposed to part of the "location" spec in the public namespace settings? Or what am I missing? Postfix has at least two books that have been proofread, but it doesn't look as if people proofread or vet the dovecot documentation wiki. Or is there something better to read than the HowTos, for a beginner? On 2012-07-21 13:07, Hans J. Albertsson wrote: > > > On 2012-07-21 12:35, Charles Marcus wrote: >> On 2012-07-21 5:24 AM, Hans J. Albertsson >> wrote: >>> Also, Sorry if I sound like a whining kid: I am new to dovecot and >>> postfix, but I was a software support engineer for Sun for ages, and I >>> should really be able to manage this if I just had some consistent and >>> non-confusing HowTo or Tutorial to follow as a first try. >> >> 1. Which 'HowTo' were you following? >> >> 2. If it wasn't the dovecot 'official' docs on the wiki, try those: >> >> http://wiki2.dovecot.org/ > I followed the > > http://wiki2.dovecot.org/ -> > HOWTOs, examples and tutorials > -> > _A simple virtual passwd file installation > _ > > HowTo. > > > >> >> $random_howtos on the internet is not the way to build a secure mail >> server. >> >> Some of the wiki2 docs still have some v1 cruft in them, but for the >> most part they should be all you need (along with asking questions >> here for clarification), and it is a wiki - anything that you >> encounter that you find confusing or incomplete, by all means come >> here for clarification, and then update the wiki to help others who >> encounter the same issues as you. >> >> You haven't even provided the most basic of details of the problem(s) >> you're having (like, for example, OS/platform, dovecot -n output, >> logs exhibiting the problem(s), etc... >> >> First, take a step back, and start with one thing at a time (testing >> and making sure each step is working as expected before moving to the >> next one)... >> >> 1. install postfix, secure it, and get it delivering mail to at least >> one (test if desired) virtual domain/users maildirs using its >> built-in virtual delivery agent, > Done, tested, works. >> >> 2. add dovecot into the mix to serve up the users mail, > And that's where I get lost: I keep fiddling back and forth wondering > where I am... > I suppose I should instead think like this: (comment, please!) > Read the HowTo > Look at the text chunks > For each element, find where that happens in conf.d, and do it > there, with the minimum change possible > And then the LDA and later LMTP. I suppose if one succeeds in getting > the LDA working, one can then move over to LMTP, with less effort? >> >> 3. switch to the dovecot delivery agent (I recommend LMTP, but at a >> minimum use the Dovecot LDA), > OK. > >> >> 4. enable dovecot-sasl, >> > >> 5. integrate dspam, >> >> 6. integrate mailman >> >> Obviously, each step requires you to know how you want things to work >> - ie, do you want to provide POP+IMAP, or only IMAPS access, etc... >> >> Last - to answer the one specific complaint that you had - dovecot >> will use all of those separate config files in the conf.d directory, >> or you can put all of your settings into one config file, the choice >> is yours - see the wiki for details: >> >> http://wiki2.dovecot.org/BasicConfiguration >> > From michael.domann at gmx.de Sat Jul 21 18:04:21 2012 From: michael.domann at gmx.de (Michael Domann) Date: Sat, 21 Jul 2012 17:04:21 +0200 Subject: [Dovecot] Upgrade Problem from 2.0.18 to 2.1.7 Message-ID: <500AC4F5.50209@gmx.de> hi there, i'm running a debian wheezy in a chroot at a fritzbox 7230, running with freetz. Running well until the upgrade of version 2.1.7 from debian repros comming up. after upgrade, the server comes up, but login is not possible. Jul 20 20:13:46 master: Warning: Killed with signal 15 (by pid=15309 uid=0 code=kill) Jul 20 20:13:46 master: Info: Dovecot v2.1.7 starting up (core dumps disabled) Jul 20 20:14:15 lda: Debug: Loading modules from directory: /usr/lib/dovecot/modules Jul 20 20:14:15 lda: Debug: Loading modules from directory: /usr/lib/dovecot/modules Jul 20 20:14:15 lda: Debug: Module loaded: /usr/lib/dovecot/modules/lib90_sieve_plugin.so Jul 20 20:14:15 lda(mdomann): Debug: Effective uid=1000, gid=1000, home=/home/mdomann Jul 20 20:14:15 lda: Debug: Module loaded: /usr/lib/dovecot/modules/lib90_sieve_plugin.so Jul 20 20:14:15 lda(mdomann): Debug: Namespace : type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mdomann/Mail:LAYOUT=fs Jul 20 20:14:15 lda(mdomann): Debug: fs: root=/home/mdomann/Mail, index=, control=, inbox=/home/mdom ann/Mail, alt= Jul 20 20:14:15 lda(mdomann): Debug: Effective uid=1000, gid=1000, home=/home/mdomann Jul 20 20:14:15 lda(mdomann): Debug: Namespace : type=private, prefix=, sep=, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mdomann/Mail:LAYOUT=fs Jul 20 20:14:15 lda(mdomann): Debug: fs: root=/home/mdomann/Mail, index=, control=, inbox=/home/mdom ann/Mail, alt= Jul 20 20:14:15 lda(mdomann): Debug: userdb lookup skipped, username taken from USER environment Jul 20 20:14:15 lda(mdomann): Debug: none: root=, index=, control=, inbox=, alt= Jul 20 20:14:15 lda(mdomann): Debug: userdb lookup skipped, username taken from USER environment Jul 20 20:14:15 lda(mdomann): Debug: none: root=, index=, control=, inbox=, alt= Jul 20 20:14:16 lda(mdomann): Debug: Destination address: mdomann at fritz.fonwlan.box (source: user at ho stname) Jul 20 20:14:16 lda(mdomann): Debug: Destination address: mdomann at fritz.fonwlan.box (source: user at ho stname) Jul 20 20:14:16 lda(mdomann): Debug: sieve: include: sieve_global_dir is not set; it is currently no t possible to include `:global' scripts. Jul 20 20:14:16 lda(mdomann): Debug: sieve: include: sieve_global_dir is not set; it is currently no t possible to include `:global' scripts. Jul 20 20:14:16 lda(mdomann): Debug: sieve: using sieve path for user's script: /home/mdomann/.dovec ot.sieve Jul 20 20:14:16 lda(mdomann): Debug: sieve: using sieve path for user's script: /home/mdomann/.dovec ot.sieve Jul 20 20:14:16 lda(mdomann): Debug: sieve: opening script /home/mdomann/.dovecot.sieve Jul 20 20:14:16 lda(mdomann): Debug: sieve: opening script /home/mdomann/.dovecot.sieve Jul 20 20:14:16 lda(mdomann): Debug: sieve: script binary /home/mdomann/.dovecot.svbin successfully loaded Jul 20 20:17:15 imap: Error: dovecot/imap: error while loading shared libraries: libpthread.so.0: failed to map segment from shared object: Cannot allocate memory Jul 20 20:17:15 imap: Fatal: master: service(imap): child 22594 returned error 127 Jul 20 20:17:15 master: Error: service(imap): command startup failed, throttling for 2 secs Jul 20 20:17:15 imap-login: Error: read(imap) failed: Connection reset by peer Jul 20 20:17:15 imap-login: Info: Internal login failure (pid=22586 id=1) (internal failure, 1 succesful auths): user=, method=PLAIN, rip=192.168.2.105, lip=192.168.2.1, TLS, session= the libary is still there fritz:~# ldd /usr/sbin/dovecot libdovecot.so.0 => /usr/lib/dovecot/libdovecot.so.0 (0x2aaaa000) libc.so.6 => /lib/mips-linux-gnu/libc.so.6 (0x2ab48000) libdl.so.2 => /lib/mips-linux-gnu/libdl.so.2 (0x2accc000) librt.so.1 => /lib/mips-linux-gnu/librt.so.1 (0x2ace0000) /lib/ld.so.1 (0x55550000) libpthread.so.0 => /lib/mips-linux-gnu/libpthread.so.0 (0x2acfa000) the ulimit i have tried, but without success. root at fritz:/var/mod/root# ulimit -a -f: file size (blocks) unlimited -t: cpu time (seconds) unlimited -d: data seg size (kb) unlimited -s: stack size (kb) unlimited -c: core file size (blocks) unlimited -m: resident set size (kb) unlimited -l: locked memory (kb) unlimited -p: processes unlimited -n: file descriptors 16384 -v: address space (kb) unlimited -w: locks unlimited fritz:~# dovecot -n # 2.0.18: /etc/dovecot/dovecot.conf # OS: Linux 2.6.28.10 mips Debian wheezy/sid squashfs base_dir = /var/run/dovecot/ default_client_limit = 100 default_process_limit = 50 default_vsz_limit = 6 M disable_plaintext_auth = no lda_mailbox_autocreate = yes listen = * log_path = /var/log/dovecot login_greeting = Dovecot on sysiphus ready. mail_debug = yes mail_gid = 1000 mail_location = maildir:/home/mdomann/Mail:LAYOUT=fs mail_uid = 1000 namespace { hidden = no inbox = yes list = yes location = prefix = subscriptions = yes type = private } passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_path = /etc/dovecot/dovecot.sieve } postmaster_address = mdomann at localhost protocols = " imap" service anvil { client_limit = 210 } service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = mdomann mode = 0777 user = mdomann } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 1 } service lmtp { unix_listener lmtp { mode = 0666 } } ssl_cert = References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> <500A8D6F.1090808@branneriet.se> <500AC18C.4030000@branneriet.se> Message-ID: <76FB3EE1-F00D-4160-B6C2-9FF0AFA0BD09@leuxner.net> Am 21.07.2012 um 16:49 schrieb Hans J. Albertsson: > location = maildir:/var/vmail/public:LAYOUT=fs:INDEX=~/public This namespace is defined for "public/shared" mailboxes <> private mailboxes: See: http://wiki2.dovecot.org/SharedMailboxes/Public Basically something different from the setting you appear to be looking for. > User Home directory structure:/var/vmail/// This is indeed the structure used in this example. The mail location is set to: mail_location = maildir:~/Maildir In this configuration this translates to the actual location being read from the userdb (dynamically): [?] args = username_format=%u /var/vmail/auth.d/%d/passwd [ file: /var/vmail/auth.d//passwd ] @:{SSHA}xxxx:5000:5000::/var/vmail///... [?] While this may not be the easiest configuration example to start with, it is a quite scalable and flexible approach though. You may find more background on this here: http://wiki2.dovecot.org/MailLocation http://wiki2.dovecot.org/UserDatabase Regards Thomas -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 203 bytes Desc: Message signed with OpenPGP using GPGMail URL: From daniel.parthey at informatik.tu-chemnitz.de Sat Jul 21 23:05:03 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 21 Jul 2012 22:05:03 +0200 Subject: [Dovecot] Dsync replication In-Reply-To: <5006E334.7010903@airstreamcomm.net> References: <4FF5C09F.2060001@airstreamcomm.net> <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> <4FF74A76.6010605@airstreamcomm.net> <43111E61-9EA3-4D18-B55B-C0F40DD891BE@iki.fi> <5006E334.7010903@airstreamcomm.net> Message-ID: <20120721200503.GA29262@daniel.localdomain> list at airstreamcomm.net wrote: > On 7/6/12 9:56 PM, Timo Sirainen wrote: > >On 6.7.2012, at 23.28, list at airstreamcomm.net wrote: > > > >>Thanks, that certainly helps identify the configuration options. However I > >>am more concerned about the experiences of others who have actually used > >>the replication. What is the rate of change on your mail cluster, how many > >>concurrent users do you support with replication enabled, do you use > >>synchronous or asynchronous replication, are you using it in an > >>active/active or active/passive state, is it possible to have a cluster > >>with multiple servers at each site hosting the same mail data, does dysnc > >>replication scale well (10,000 -> 100,000 -> 1,000,000 users)? Just trying > >>to get a good feel for whether dsync replication is capable of handling the > >>use case I am proposing before investing too much time in testing it. > > > >I wouldn't use it for large systems yet. It is still pretty inefficient. > >v2.2 will have a redesigned dsync that can do incremental syncs much faster > >and with less bandwidth. Anyway, in my small installation I'm using it in > >active-active mode and it works well enough. I've even configured my clients > >intentionally so that they use different servers. > > > Does dsync replication only work between two hosts? In my scenario > I would have two sites with X number of nodes at each with an NFS > backend for each site. For this example lets say I have site A with > two nodes that mount one NFS share, and site B with two nodes that > mount one NFS share. Is it possible to implement dsync replication > between these two clusters of nodes? If you respect http://wiki2.dovecot.org/NFS and setup a director http://wiki2.dovecot.org/Director including a doveadm proxy it could work. dsync is part of "doveadm backup", so you just have to get your director setup right so that imap, pop3, lmtp and doveadm services are always proxied to the correct nfs-client-node at the local site. Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Jul 22 00:04:02 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 21 Jul 2012 23:04:02 +0200 Subject: [Dovecot] Upgrade Problem from 2.0.18 to 2.1.7 In-Reply-To: <500AC4F5.50209@gmx.de> References: <500AC4F5.50209@gmx.de> Message-ID: <20120721210402.GA29891@daniel.localdomain> Michael Domann wrote: > Jul 20 20:17:15 imap: Error: dovecot/imap: error while loading shared libraries: libpthread.so.0: failed to map segment from shared object: Cannot allocate memory > Jul 20 20:17:15 imap: Fatal: master: service(imap): child 22594 returned error 127 Looks like a "out of memory" problem. > default_vsz_limit = 6 M Maybe increase this value. On my Linux box it is set to 256M. Possibly 6M is not enough? > root at fritz:/var/mod/root# free -m > total used free shared buffers > Mem: 59512 52260 7252 0 2452 > -/+ buffers: 49808 9704 > Swap: 538608 8116 530492 You're almost running out of memory, used 50MB out of 60MB. Try to temporarily disable other services on the box and see the IMAP service runs smoothly? Regards, Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sun Jul 22 00:18:13 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 21 Jul 2012 23:18:13 +0200 Subject: [Dovecot] Director for smtp-auth In-Reply-To: <500A8445.4060601@wk-serv.de> References: <500A8445.4060601@wk-serv.de> Message-ID: <20120721211813.GA30245@daniel.localdomain> Patrick Westenberg wrote: > isn't it possible to use the director functionality for smtp-auth too? > > I have a frontend system for imap and outgoing mails using sasl and > I recognized that all sasl username/password combinations are > accepted as these credentials aren't checked by the backend servers. You could configure saslauthd to use "rimap" (remote imap) as authentication backend and then use your existing dovecot imap director infrastructure. Regards Daniel -- https://plus.google.com/103021802792276734820 From h.reindl at thelounge.net Sun Jul 22 00:30:31 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Sat, 21 Jul 2012 23:30:31 +0200 Subject: [Dovecot] Upgrade Problem from 2.0.18 to 2.1.7 In-Reply-To: <20120721210402.GA29891@daniel.localdomain> References: <500AC4F5.50209@gmx.de> <20120721210402.GA29891@daniel.localdomain> Message-ID: <500B1F77.5000502@thelounge.net> Am 21.07.2012 23:04, schrieb Daniel Parthey: > Michael Domann wrote: >> Jul 20 20:17:15 imap: Error: dovecot/imap: error while loading shared libraries: libpthread.so.0: failed to map segment from shared object: Cannot allocate memory >> Jul 20 20:17:15 imap: Fatal: master: service(imap): child 22594 returned error 127 > > Looks like a "out of memory" problem. > >> default_vsz_limit = 6 M > Maybe increase this value. On my Linux box it is set to 256M. > Possibly 6M is not enough? yes, this is way too few >> root at fritz:/var/mod/root# free -m >> total used free shared buffers >> Mem: 59512 52260 7252 0 2452 >> -/+ buffers: 49808 9704 >> Swap: 538608 8116 530492 > > You're almost running out of memory, used 50MB out of 60MB you missed the param "-m" -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From claus.rosenberger at rocnet.de Sun Jul 22 01:04:19 2012 From: claus.rosenberger at rocnet.de (Claus Rosenberger) Date: Sat, 21 Jul 2012 22:04:19 +0000 Subject: [Dovecot] Panic: file mail-storage.c: line 791 (mailbox_check_mismatching_separators) Message-ID: <1184100f1800b1ecd29e7f08f873e026c1596784@rocnet.de> Hi, with fetching the folder structure with the webclient Tine i can see following error messages inside of the logfiles: Panic: file mail-storage.c: line 791 (mailbox_check_mismatching_separators): assertion failed: (strncmp(vname, ns->prefix, ns->prefix_len-1) == 0) Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x448fa) [0x7f70036db8fa] -> /usr/lib/dovecot/libdovecot.so.0(+0x44946) [0x7f70036db946] -> /usr/lib/dovecot/libdovecot.so.0(+0x1b8ca) [0x7f70036b28ca] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_open_stream+0) [0x7f70039a1550] -> /usr/lib/dovecot/libdovecot-storage.so.0(mailbox_open+0xf) [0x7f70039a156f] -> dovecot/imap(cmd_select_full+0x14a) [0x40db1a] -> dovecot/imap(command_exec+0x3d) [0x41160d] -> dovecot/imap() [0x41058e] -> dovecot/imap() [0x41066d] -> dovecot/imap(client_handle_input+0x135) [0x410895] -> dovecot/imap(client_input+0x5f) [0x4111bf] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f70036e7ea6] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x9f) [0x7f70036e8f2f] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7f70036e7e48] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f70036d4103] -> dovecot/imap(main+0x2a5) [0x419695] -> /lib/libc.so.6(__libc_start_main+0xfd) [0x7f7003353c8d] -> dovecot/imap() [0x408f49] Fatal: master: service(imap): child 21152 killed with signal 6 (core dumps disabled) ------------ dovecot -n ------------ # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 ext4 first_valid_gid = 104 first_valid_uid = 107 last_valid_gid = 104 last_valid_uid = 107 login_trusted_networks = 10.20.11.4 mail_gid = 104 mail_location = maildir:/var/spool/dovecot/%Ld/%Ln/Maildir mail_plugins = quota acl mail_privileged_group = mail mail_uid = 107 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { ? hidden = no ? list = children ? location = maildir:/var/spool/dovecot/%%d/%%n/Maildir:INDEX=/var/spool/dovecot/%d/%n/Maildir/shared/%%n ? prefix = shared/%%n/ ? separator = / ? type = shared } namespace inbox { ? inbox = yes ? location = ? prefix = INBOX/ ? separator = / ? type = private } namespace virtual { ? alias_for = INBOX/ ? hidden = yes ? list = no ? location = ? prefix = ? separator = / ? type = private } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } plugin { ? acl = vfile ? acl_shared_dict = file:/var/spool/dovecot/%d/shared-mailboxes ? quota = maildir:User quota ? quota_rule = *:storage=1GB ? sieve = /var/spool/dovecot/%Ld/%Ln/.dovecot.sieve ? sieve_dir = /var/spool/dovecot/%Ld/%Ln/sieve } postmaster_address = postmaster at rocnet.de protocols = " imap sieve pop3 sieve" service auth { ? unix_listener auth-userdb { ??? group = dovecot ??? mode = 0660 ??? user = vmail ? } } service imap-login { ? inet_listener imap { ??? port = 0 ? } ? inet_listener imaps { ??? port = 993 ??? ssl = yes ? } } service imap { ? process_limit = 1024 } service pop3-login { ? inet_listener pop3 { ??? port = 0 ? } ? inet_listener pop3s { ??? port = 995 ??? ssl = yes ? } } service pop3 { ? process_limit = 1024 } ssl_cert = References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> <500A8D6F.1090808@branneriet.se> <500AC18C.4030000@branneriet.se> <76FB3EE1-F00D-4160-B6C2-9FF0AFA0BD09@leuxner.net> Message-ID: <500BCA3D.30807@branneriet.se> I find your answer a bit confusing: I was showing these problems off as sources of confusion, not as examples of what I want to achieve! My view of what a HowTo or a Tutorial is supposed to be is: To the point: Explain what you are going to demonstrate, and explain to what extent it can or cannot serve as a boiler-plate for more advanced configs. Complete ( in the appropriate manner ): For every single thing you display, explain what the reader is supposed to do about it, and make sure you explain what is going to appear automagically and what the reader must do to achieve what won't appear as a result of dovecot's own actions. Do not point the reader to other docs unless those docs agree with the current one in both form and perspective. If they don't, copy in and adjust to fit the current doc. Many of the "other docs" pointed to by the HowTos (and there appears to be no tutorials at all ) leave the uninitiated confused, because they appear to make slightly different assumptions from the referring doc. Many docs say "you must first configure THIS," LMTP refers you back to the LDA config, and that may be absolutely correct, but me not being an initiate, I get lost. Because it seems that the perspective and the assumptions are slightly skewed or totally different, how can I tell at the first attempt? I started out doing postfix config: that was easy, very easy, and I have set up several different ones now, none of which took more than an hour. The first one was Chap 3 in Book Of Postfix by the German fellows, and that is just about the best tutorial I have ever seen. I did that w/o understanding the first thing about postfix vs sendmail,and it taught me a lot, and I could then read the rest of the book much faster. On 2012-07-21 21:00, Thomas Leuxner wrote: > Am 21.07.2012 um 16:49 schrieb Hans J. Albertsson: > >> location = maildir:/var/vmail/public:LAYOUT=fs:INDEX=~/public > This namespace is defined for "public/shared" mailboxes<> private mailboxes: > > See: http://wiki2.dovecot.org/SharedMailboxes/Public > > Basically something different from the setting you appear to be looking for. > >> User Home directory structure:/var/vmail/// > This is indeed the structure used in this example. The mail location is set to: > > mail_location = maildir:~/Maildir > > In this configuration this translates to the actual location being read from the userdb (dynamically): > > [?] > args = username_format=%u /var/vmail/auth.d/%d/passwd > > [ file: /var/vmail/auth.d//passwd ] > @:{SSHA}xxxx:5000:5000::/var/vmail///... > [?] > > While this may not be the easiest configuration example to start with, it is a quite scalable and flexible approach though. You may find more background on this here: > > http://wiki2.dovecot.org/MailLocation > http://wiki2.dovecot.org/UserDatabase > > Regards > Thomas > From hans.j.albertsson at branneriet.se Sun Jul 22 12:53:37 2012 From: hans.j.albertsson at branneriet.se (Hans J. Albertsson) Date: Sun, 22 Jul 2012 11:53:37 +0200 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver In-Reply-To: <500BCA3D.30807@branneriet.se> References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> <500A8D6F.1090808@branneriet.se> <500AC18C.4030000@branneriet.se> <76FB3EE1-F00D-4160-B6C2-9FF0AFA0BD09@leuxner.net> <500BCA3D.30807@branneriet.se> Message-ID: <500BCDA1.7010202@branneriet.se> On 2012-07-22 11:39, Hans J. Albertsson wrote: > > > To the point: Explain what you are going to demonstrate, and explain > to what extent it can or cannot serve as a boiler-plate for more > advanced configs. > Complete ( in the appropriate manner ): For every single thing you > display, explain what the reader is supposed to do about it, and make > sure you explain what is going to appear automagically and what the > reader must do to achieve what won't appear as a result of dovecot's > own actions. > Do not point the reader to other docs unless those docs agree with the > current one in both form and perspective. If they don't, copy in and > adjust to fit the current doc. > > And also, make sure that if you write in english, the meaning of your words in absolutely unequivocal, there must not appear in any reader's mind the slightest doubt as to how (s)he's supposed to use the info given. As an example, to wit, in the http://wiki2.dovecot.org/SharedMailboxes/Public doc, there's a line "In the above example, you would then create Maildir mailboxes under the /var/mail/public/ directory." and a colour plate plate showing a directory listing. # ls -la /var/mail/public/ drwxr-s--- 1 root mail 0 2007-03-19 03:12 . drwxrws--- 1 root mail 0 2007-03-19 03:12 .lkml drwxrws--- 1 root mail 0 2007-03-19 03:12 .bugtraq -rw-rw---- 1 root mail 0 2007-03-19 03:12 dovecot-shared I am guessing that this means I'm supposed to do mkdir dovecot-shared inside /var/mail/public. But "creating Maildir mailboxes" might mean more than just mkdir, and not explaining that bit at this point in the doc slows the reader down, especially if (s)he's not already well versed in the mysteries of dovecot wizardry. And if (s)he is that, why should (s)he read the doc at all? Sorry if I'm being horridly difficult, but I think (from experiencing it as a user) dovecot is too good not to have proper tutorials and howtos. From CMarcus at Media-Brokers.com Sun Jul 22 13:41:26 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Sun, 22 Jul 2012 06:41:26 -0400 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver In-Reply-To: <500BCDA1.7010202@branneriet.se> References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> <500A8D6F.1090808@branneriet.se> <500AC18C.4030000@branneriet.se> <76FB3EE1-F00D-4160-B6C2-9FF0AFA0BD09@leuxner.net> <500BCA3D.30807@branneriet.se> <500BCDA1.7010202@branneriet.se> Message-ID: <500BD8D6.1060409@Media-Brokers.com> On 2012-07-22 5:53 AM, Hans J. Albertsson > As an example, to wit, in the > http://wiki2.dovecot.org/SharedMailboxes/Public doc, there's a line > > "In the above example, you would then create Maildir mailboxes under the > /var/mail/public/ directory." > > and a colour plate plate showing a directory listing. > > # ls -la /var/mail/public/ > drwxr-s--- 1 root mail 0 2007-03-19 03:12 . > drwxrws--- 1 root mail 0 2007-03-19 03:12 .lkml > drwxrws--- 1 root mail 0 2007-03-19 03:12 .bugtraq > -rw-rw---- 1 root mail 0 2007-03-19 03:12 dovecot-shared > > I am guessing that this means I'm supposed to do mkdir dovecot-shared > inside /var/mail/public. Since it isn't listed as a directory, I'm confused as to why would you guess that? dovecot-shared is a FILE, not a directory. The 3rd line below that example on that page specifically says: "The dovecot-shared FILE..." It seems to me that you aren't even bothering to read these docs, andit is more like all you want to do is complain that there is nothing already written holding your hand through every possible config that you want to accomplish. Dovecot is primarily written by one guy (Timo), and he does a remarkable job of both coding and documenting dovecot on the wiki, as well as answering support questions here on the list, and while sometimes there are a few days before he answers many questions, serious bug reports generally get prompt attention, and I don't think I've ever seen him not respond to a question in time. There is no doubt that dovecot could really use some good, experienced technical writers that could help Timo with documenting dovecot to make it easier to learn by someone new to it, and I'm sure he would welcome that help - are you volunteering? > Sorry if I'm being horridly difficult, but I think (from experiencing it > as a user) dovecot is too good not to have proper tutorials and howtos. Well, dovecot's intended audience isn't a 'user', it is experienced system/mail admins, but if you are volunteering to help Timo (and the dovecot community) out by improving the wiki documentation and/or creating some of these HowTos from the perspective of someone totally new to dovecot (and maybe even IMAP servers in general), then I am quite certain that Timo will welcome such help. And as for documentation in the form of books, you cannot compare dovecot to postfix in this regard. Postfix is one of the most mature, stable projects out there - it's core functionality basically never changes (only the rare bug fixes), and major new features are pretty rare too, so even books written 8 years ago are still fairly relevant (and generally are only missing the new features). With dovecot, things are very different. It is still very young and changing rapidly, and probably will continue to do so as Timo adds new features on his ToDo list. A book written even a year ago would not have much use to someone using the current version today. As it matures and features stabilize, this will change, and I'm hopeful that in a year or two, dovecot will stabilize to the point that some of the talented book writers out there will take on such a huge project - but none of them want to do that right now because dovecot is such a fast moving target. -- Best regards, Charles From hans.j.albertsson at branneriet.se Sun Jul 22 15:49:19 2012 From: hans.j.albertsson at branneriet.se (Hans J. Albertsson) Date: Sun, 22 Jul 2012 14:49:19 +0200 Subject: [Dovecot] Confusion when trying to set up a first postfix+dovecot mailserver In-Reply-To: <500BD8D6.1060409@Media-Brokers.com> References: <500A7539.7060403@branneriet.se> <500A85D7.7030900@Media-Brokers.com> <500A8D6F.1090808@branneriet.se> <500AC18C.4030000@branneriet.se> <76FB3EE1-F00D-4160-B6C2-9FF0AFA0BD09@leuxner.net> <500BCA3D.30807@branneriet.se> <500BCDA1.7010202@branneriet.se> <500BD8D6.1060409@Media-Brokers.com> Message-ID: <500BF6CF.3020202@branneriet.se> Sorry, you're right: I was stressed out when writing this. I meant mkdir .lkml and mkdir .bugtraq and touch dovecot-shared. Of course. It is not the best example, I agree. Still it should say something like "create the file dovecot-shared if you want a shared mailbox named dovecot-shared" (can probably be whittled down). And, yes, if I can get thru setting up dovecot properly ( I know I can, it will just take longer ) I will of course do a writeup in the style I want to see myself. When it comes to dovecot, I am a user (that IS a terribly derogative term, isn't it?), but I have been actively supporting large farms of servers running other forms of mail delivery agents in Sun's various OSes since 1986 up to 2008, so I do think I ought to be able to get my head around dovecot, too. I will just have to read the wiki thru, all of it... :-) On 2012-07-22 12:41, Charles Marcus wrote: > On 2012-07-22 5:53 AM, Hans J. Albertsson > As an example, to wit, in the >> http://wiki2.dovecot.org/SharedMailboxes/Public doc, there's a line >> >> "In the above example, you would then create Maildir mailboxes under the >> /var/mail/public/ directory." >> >> and a colour plate plate showing a directory listing. >> >> # ls -la /var/mail/public/ >> drwxr-s--- 1 root mail 0 2007-03-19 03:12 . >> drwxrws--- 1 root mail 0 2007-03-19 03:12 .lkml >> drwxrws--- 1 root mail 0 2007-03-19 03:12 .bugtraq >> -rw-rw---- 1 root mail 0 2007-03-19 03:12 dovecot-shared >> >> I am guessing that this means I'm supposed to do mkdir dovecot-shared >> inside /var/mail/public. > > Since it isn't listed as a directory, I'm confused as to why would you > guess that? > > dovecot-shared is a FILE, not a directory. > > The 3rd line below that example on that page specifically says: > > "The dovecot-shared FILE..." > > It seems to me that you aren't even bothering to read these docs, > andit is more like all you want to do is complain that there is > nothing already written holding your hand through every possible > config that you want to accomplish. > > Dovecot is primarily written by one guy (Timo), and he does a > remarkable job of both coding and documenting dovecot on the wiki, as > well as answering support questions here on the list, and while > sometimes there are a few days before he answers many questions, > serious bug reports generally get prompt attention, and I don't think > I've ever seen him not respond to a question in time. > > There is no doubt that dovecot could really use some good, experienced > technical writers that could help Timo with documenting dovecot to > make it easier to learn by someone new to it, and I'm sure he would > welcome that help - are you volunteering? > >> Sorry if I'm being horridly difficult, but I think (from experiencing it >> as a user) dovecot is too good not to have proper tutorials and howtos. > > Well, dovecot's intended audience isn't a 'user', it is experienced > system/mail admins, but if you are volunteering to help Timo (and the > dovecot community) out by improving the wiki documentation and/or > creating some of these HowTos from the perspective of someone totally > new to dovecot (and maybe even IMAP servers in general), then I am > quite certain that Timo will welcome such help. > > And as for documentation in the form of books, you cannot compare > dovecot to postfix in this regard. > > Postfix is one of the most mature, stable projects out there - it's > core functionality basically never changes (only the rare bug fixes), > and major new features are pretty rare too, so even books written 8 > years ago are still fairly relevant (and generally are only missing > the new features). > > With dovecot, things are very different. It is still very young and > changing rapidly, and probably will continue to do so as Timo adds new > features on his ToDo list. A book written even a year ago would not > have much use to someone using the current version today. As it > matures and features stabilize, this will change, and I'm hopeful that > in a year or two, dovecot will stabilize to the point that some of the > talented book writers out there will take on such a huge project - but > none of them want to do that right now because dovecot is such a fast > moving target. > From hans.j.albertsson at branneriet.se Sun Jul 22 16:39:04 2012 From: hans.j.albertsson at branneriet.se (Hans J. Albertsson) Date: Sun, 22 Jul 2012 15:39:04 +0200 Subject: [Dovecot] Performance based choices Message-ID: <500C0278.1030006@branneriet.se> I've stopped trying to find a HowTo that suits me right away, and instead I am happily trudging thru the Dovecot wiki, article by article. I have right up front thought of one question, a general one, and some detailed versions of that same question: Generally, is there much general performance and reliability background data available for making the basic choices? Say: 1st. delivery: maildrop or LDA or LMTP? I tend to think LMTP should be the ideal choice for me, from what I've gleaned so far. Above all it means that there's only one conceptual thing that needs to know, and postfix and other stuff can safely let dovecot deal with accessing and finding mailboxes. And it won't start and stop thousands of processes. 2nd. mailbox format: Maildir and mbox are the older forms. Are there any advantages to using dovecot's dbox instead? mdbox strikes me as having the potential for being a fast and reliable format. Is that an accurate impression? And, is mdbox mature enough for me to forget maildir and mbox? 3rd. I'm aiming for a poor man's High Availability system: I'm using zfs, and I'm hoping to place all config data for dovecot and postfix and everything else in one zfs file system, and all the user owned data (what should normally go in a home directory) in another (or two or three other) zfs file systems. Then I'm planning to copy over all data at regular intervals, to a second, normally passive, mail server. If the main server breaks, we'll manually (or using scripts and autodetection) fail over to the passive one, making it active, and turning power off to the failed guy using IPMI. The data transfer is to be zfs send/zfs recv over a separate highly redundant network connection. Is this a reasonable idea, or is there some advantage to letting dsync do some of the copying??? Or is there some totally different alternative? iscsi? 4th. With about a thousand users/accounts: does MySQL pay off? Or is LDAP the way to go? Or will a dovecot-specific passwd-file do the job well enough? Those are the three I'm used to since before. I'd like to stay with the flat file, but not the system password file: we're not going to let users in except into dovecot. From michael.domann at gmx.de Sun Jul 22 17:46:33 2012 From: michael.domann at gmx.de (Michael Domann) Date: Sun, 22 Jul 2012 16:46:33 +0200 Subject: [Dovecot] Upgrade Problem from 2.0.18 to 2.1.7 In-Reply-To: <20120721210402.GA29891@daniel.localdomain> References: <500AC4F5.50209@gmx.de> <20120721210402.GA29891@daniel.localdomain> Message-ID: <500C1249.6050702@gmx.de> Thanks for pointing the right direction. I have set default_vsz_limit = 16M and it works. And a short question, possible i should make a new thread, but i think it's a little config issue from me. Can i prevent dovecot from making new dirs? Because i have one draft and one entw?rfe dir, also one trash and one papierkorb. one german, one english. the dirs are always created new, after i delete them. thanks and regards Michael Am 21.07.2012 23:04, schrieb Daniel Parthey: > Michael Domann wrote: >> Jul 20 20:17:15 imap: Error: dovecot/imap: error while loading shared libraries: libpthread.so.0: failed to map segment from shared object: Cannot allocate memory >> Jul 20 20:17:15 imap: Fatal: master: service(imap): child 22594 returned error 127 > > Looks like a "out of memory" problem. > >> default_vsz_limit = 6 M > > Maybe increase this value. On my Linux box it is set to 256M. > Possibly 6M is not enough? > >> root at fritz:/var/mod/root# free -m >> total used free shared buffers >> Mem: 59512 52260 7252 0 2452 >> -/+ buffers: 49808 9704 >> Swap: 538608 8116 530492 > > You're almost running out of memory, used 50MB out of 60MB. > > Try to temporarily disable other services on the box and > see the IMAP service runs smoothly? > > Regards, > Daniel From daniel.parthey at informatik.tu-chemnitz.de Sun Jul 22 18:23:11 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 22 Jul 2012 17:23:11 +0200 Subject: [Dovecot] Upgrade Problem from 2.0.18 to 2.1.7 In-Reply-To: <500C1249.6050702@gmx.de> References: <500AC4F5.50209@gmx.de> <20120721210402.GA29891@daniel.localdomain> <500C1249.6050702@gmx.de> Message-ID: <20120722152311.GA11986@daniel.localdomain> Michael Domann wrote: > Can i prevent dovecot from making new dirs? > Because i have one draft and one entw?rfe dir, also one trash and > one papierkorb. one german, one english. the dirs are always created > new, after i delete them. Most probably, your mail clients are responsible for creating the folders via IMAP. Dovecot only performs the commands from your IMAP client, so this is rather a client issue. You will need to tell *all* your mail user agents and your mobile phone to use the same folders for the same purpose. Regards Daniel -- https://plus.google.com/103021802792276734820 From yeupou at gnu.org Sun Jul 22 14:46:57 2012 From: yeupou at gnu.org (Mathieu Roy) Date: Sun, 22 Jul 2012 13:46:57 +0200 Subject: [Dovecot] dovecot 2.x via xinetd Message-ID: <201207221346.57165@bender.stalag13.ici> Hello, I was using dovecot 1.2.x via xinetd with a setup like http://wiki.dovecot.org/InetdInstall Since I upgraded to Debian Wheezy, shipping dovecot 2.x, it no longer works. At best, I end up with stuff like Error: net_connect_unix(anvil) failed: No such file or directory Fatal: Couldn't connect to anvil Any clues? Regards, (add me in Cc in replies, I'm not suscribed to the list) # doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-3-amd64 x86_64 Debian wheezy/sid ext4 mail_location = maildir:/home/%n/.Maildir:LAYOUT=fs mail_privileged_group = mail namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } protocols = " imap" ssl_cert = Hi, I'm trying to get the so-called "single instance store" (I think cyrus has got the name for the first time) with dovecot --version = 2.0.19 binary package installed from ubuntu 12.04 lts official repo. I have checked that "maildir_copy_with_hardlinks" is enabled ("dovecot -a|grep hard" shows "yes") then I have installed and enabled the lmtp component of dovecot. The configuration "dovecot -n" is pasted here: http://paste.lug.ro/131180 Also in the same paste is a strace against dovecot and childrent showing evidence of the MTA delivering a single copy of the message via LMTP with multiple RCPT TO: headers. However when looking in the Maildir, I see the mail break down into three separate files instead of expected hardlinked files ("stat" and "ls" shows one single link count, inodes are different) Given the above data, what (am I | dovecot is) doing wrong? Please cc me if you need additional input when replying as I'm not subscribed to the list (I'll watch the thread online only) Many thanks in advance. From tss at iki.fi Sun Jul 22 22:04:17 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 22 Jul 2012 22:04:17 +0300 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: <201207221346.57165@bender.stalag13.ici> References: <201207221346.57165@bender.stalag13.ici> Message-ID: <48EA2C62-814F-43CC-8D9A-FC8CD28E41A9@iki.fi> On 22.7.2012, at 14.46, Mathieu Roy wrote: > Hello, > > I was using dovecot 1.2.x via xinetd with a setup like http://wiki.dovecot.org/InetdInstall > Since I upgraded to Debian Wheezy, shipping dovecot 2.x, it no longer works. At best, I end up with stuff like > > Error: net_connect_unix(anvil) failed: No such file or directory > Fatal: Couldn't connect to anvil > > Any clues? > Regards, > > (add me in Cc in replies, I'm not suscribed to the list) Doesn't work anymore. No plans to make it work anymore. Way too much trouble. From daniel.parthey at informatik.tu-chemnitz.de Sun Jul 22 22:06:41 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sun, 22 Jul 2012 21:06:41 +0200 Subject: [Dovecot] Corrupted dbox file - purging found mismatched offsets Message-ID: <20120722190641.GA14732@daniel.localdomain> Hi, we are running a cluster using mdbox on NFS with director+mailbox on each node. After running our daily doveadm purge loop over all users, we got dbox corruption... doveadm user "*" |\ while read username do doveadm -c /etc/dovecot-director/dovecot-director.conf -D purge -u "$username" done For one user (which deleted a lot of messages from his mailboxes today), we got the following error in the dovecot.log: Jul 22 20:10:36 10.129.3.213 dovecot: doveadm(user at example.org): Error: Corrupted dbox file /mail/dovecot/example.org/user/mail/storage/m.24 (around offset=1380859): purging found mismatched offsets (1380829 vs 920527, 3/61) Jul 22 20:10:36 10.129.3.213 dovecot: doveadm(user at example.org): Warning: mdbox /mail/dovecot/example.org/user/mail/storage: rebuilding indexes What does this message exactly tell me and how to prevent this in the future? Now the indexes are broken and flags are lost. The "storage" directory content of the affected user looks as follows: drwx------ 2 vmail vmail 4096 2012-07-22 20:10 . drwx------ 4 vmail vmail 4096 2012-05-06 12:16 .. -rw------- 1 vmail vmail 99008 2012-07-22 20:10 dovecot.map.index -rw------- 1 vmail vmail 33016 2012-07-22 20:10 dovecot.map.index.log -rw------- 1 vmail vmail 34664 2012-07-22 20:10 dovecot.map.index.log.2 -rw------- 1 vmail vmail 8134 2012-05-16 11:36 m.22 -rw------- 1 vmail vmail 10572466 2012-06-02 23:09 m.24 -rw------- 1 vmail vmail 1380205 2012-05-27 10:59 m.24.broken -rw------- 1 vmail vmail 6506837 2012-06-09 15:42 m.25 -rw------- 1 vmail vmail 18201641 2012-06-17 22:50 m.26 -rw------- 1 vmail vmail 11345293 2012-07-01 14:37 m.28 -rw------- 1 vmail vmail 36707787 2012-07-08 23:05 m.29 -rw------- 1 vmail vmail 6062419 2012-07-15 23:34 m.30 -rw------- 1 vmail vmail 52396198 2012-07-22 20:10 m.31 -rw------- 1 vmail vmail 3067862 2012-07-20 14:31 m.32 -rw------- 1 vmail vmail 44520965 2012-07-20 14:31 m.33 -rw------- 1 vmail vmail 52426605 2012-07-22 20:10 m.34 -rw------- 1 vmail vmail 28984844 2012-07-22 20:10 m.35 -rw------- 1 vmail vmail 14465248 2012-07-22 20:10 m.36 -rw------- 1 vmail vmail 37451127 2012-07-22 20:10 m.37 -rw------- 1 vmail vmail 41494033 2012-07-22 20:10 m.38 -rw------- 1 vmail vmail 52066924 2012-07-22 20:10 m.39 -rw------- 1 vmail vmail 49785529 2012-07-22 20:10 m.40 -rw------- 1 vmail vmail 23509886 2012-07-22 20:10 m.41 -rw------- 1 vmail vmail 29339462 2012-07-22 20:10 m.42 -rw------- 1 vmail vmail 29510420 2012-07-22 20:10 m.43 -rw------- 1 vmail vmail 50896380 2012-07-22 20:10 m.44 -rw------- 1 vmail vmail 8331046 2012-07-22 20:10 m.45 -rw------- 1 vmail vmail 5903744 2012-07-22 20:10 m.46 -rw------- 1 vmail vmail 70281 2012-07-22 20:10 m.47 -rw------- 1 vmail vmail 27397909 2012-07-22 20:10 m.48 -rw------- 1 vmail vmail 3893509 2012-07-22 20:10 m.49 -rw------- 1 vmail vmail 1410097 2012-07-22 20:10 m.50 -rw------- 1 vmail vmail 1905759 2012-07-22 20:10 m.51 -rw------- 1 vmail vmail 51783968 2012-05-06 12:12 m.9 Any hints on how to prevent dbox corruption in this case? And what should I do with the "m.24.broken" file now? Regards Daniel -- https://plus.google.com/103021802792276734820 -------------- next part -------------- # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_verbose = yes auth_verbose_passwords = sha1 deliver_log_format = mailbox: deliver: msgid=%m from=%f: %$ dict { quota = mysql:/etc/dovecot/conf.d/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no doveadm_password = xxx imapc_features = rfc822.size imapc_host = local-mailbox imapc_port = 18143 instance_name = dovecot-mailbox lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes login_greeting = Mailbox login_log_format = mailbox: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "mailbox: mail: %s(%u): " mail_plugins = quota stats mail_privileged_group = vmail mail_uid = vmail managesieve_implementation_string = Sieve managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 weeks mdbox_rotate_size = 50 M mmap_disable = yes passdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } plugin { quota = dict:User quota::proxy::quota quota_rule = *:storage=10G quota_rule2 = Trash:storage=+100M quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve stats_refresh = 30 secs stats_track_cmds = yes } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { group = dovecot mode = 0660 user = dovecot } } service dict { unix_listener dict { group = vmail mode = 0660 } } service doveadm { inet_listener doveadm-server { port = 19000 } } service imap-login { inet_listener imap { port = 19143 } } service imap-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service imap { executable = imap imap-postlogin } service lmtp { inet_listener lmtp { address = * port = 19024 } } service managesieve-login { inet_listener sieve { port = 19200 } } service pop3-login { inet_listener pop3 { port = 19110 } } service pop3-postlogin { executable = script-login /usr/local/bin/dovecot-postlogin user = $default_internal_user } service pop3 { executable = pop3 pop3-postlogin } service quota-warning { executable = script /usr/local/bin/quota-warning extra_groups = dovecot unix_listener quota-warning { user = vmail } user = vmail } service stats { fifo_listener stats-mail { mode = 0600 user = vmail } } ssl = no userdb { driver = prefetch } userdb { args = /etc/dovecot/conf.d/dovecot-sql.conf.ext driver = sql } verbose_proctitle = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_plugins = quota stats imap_quota imap_stats } protocol lmtp { mail_plugins = quota stats sieve } -------------- next part -------------- # 2.1.8: /etc/dovecot-director/dovecot-director.conf # OS: Linux 2.6.32-40-server x86_64 Ubuntu 10.04.4 LTS auth_verbose = yes auth_verbose_passwords = sha1 base_dir = /var/run/dovecot-director deliver_log_format = director: deliver: msgid=%m from=%f: %$ director_doveadm_port = 20000 director_mail_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_servers = 10.129.3.193 10.129.3.192 10.129.3.191 10.129.3.190 director_user_expire = 2 days disable_plaintext_auth = no doveadm_password = xxx doveadm_proxy_port = 19000 instance_name = dovecot-director lmtp_proxy = yes login_greeting = Mail Balancer login_log_format = director: login: %$: %s login_trusted_networks = 10.129.3.0/24 mail_debug = yes mail_fsync = always mail_gid = vmail mail_home = /mail/dovecot/%d/%n mail_location = mdbox:~/mail mail_log_prefix = "director: mail: %s(%u): " mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /etc/dovecot-director/conf.d/dovecot-sql.conf.ext driver = sql } protocols = imap pop3 lmtp sieve service auth { unix_listener auth-userdb { user = dovecot } } service director { fifo_listener login/proxy-notify { mode = 0666 } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0600 } unix_listener login/director { mode = 0666 } } service doveadm { executable = doveadm-server director inet_listener doveadm-server { port = 20000 } } service imap-login { executable = imap-login director inet_listener imap { port = 20143 } inet_listener imaps { port = 20993 ssl = yes } } service lmtp { inet_listener lmtp { address = * port = 20024 } } service managesieve-login { executable = managesieve-login director inet_listener sieve { port = 20200 } } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 20110 } inet_listener pop3s { port = 20995 ssl = yes } } ssl_cert = References: <500C0278.1030006@branneriet.se> Message-ID: <500C623F.7090202@schetterer.org> Am 22.07.2012 15:39, schrieb Hans J. Albertsson: > I've stopped trying to find a HowTo that suits me right away, and > instead I am happily trudging thru the Dovecot wiki, article by article. > > I have right up front thought of one question, a general one, and some > detailed versions of that same question: > > Generally, is there much general performance and reliability background > data available for making the basic choices? > > Say: 1st. > delivery: > maildrop or LDA or LMTP? lmtp should be fine > I tend to think LMTP should be the ideal choice for me, from what I've > gleaned so far. > Above all it means that there's only one conceptual thing that needs to > know, and postfix and other stuff can safely let dovecot deal with > accessing and finding mailboxes. > And it won't start and stop thousands of processes. > > 2nd. > mailbox format: Maildir and mbox are the older forms. Are there any > advantages to using dovecot's dbox instead? mdbox strikes me as having > the potential for being a fast and reliable format. Is that an accurate > impression? And, is mdbox mature enough for me to forget maildir and mbox? mdbox maybe the best choice > > 3rd. > I'm aiming for a poor man's High Availability system: > I'm using zfs, and I'm hoping to place all config data for dovecot and > postfix and everything else in one zfs file system, and all the user > owned data (what should normally go in a home directory) in another (or > two or three other) zfs file systems. > Then I'm planning to copy over all data at regular intervals, to a > second, normally passive, mail server. If the main server breaks, we'll > manually (or using scripts and autodetection) fail over to the passive > one, making it active, and turning power off to the failed guy using IPMI. > The data transfer is to be zfs send/zfs recv over a separate highly > redundant network connection. for poor man i would recommend master/master drbd and some cluster filesystem i.e ocfs2 etc, for backup you may use dsync with loadbalancing i.e keepalived etc anyway you need a "quick" storage for imap a standby soltution maybe good too, but why not simply use loadbalancing > Is this a reasonable idea, or is there some advantage to letting dsync > do some of the copying??? Or is there some totally different > alternative? iscsi? > > 4th. > With about a thousand users/accounts: does MySQL pay off? Or is LDAP the > way to go? Or will a dovecot-specific passwd-file do the job well enough? > Those are the three I'm used to since before. > I'd like to stay with the flat file, but not the system password file: > we're not going to let users in except into dovecot. with modern hardware/kernel cpu and mem thousends of user accounts are no problem with mysql, but i guess same is in ldap anyway this is only my meaning, wait till you heard more ones before act Perhaps you should give more additional info like planned standard quota of mailboxes awaited average user cons etc for better advice -- Best Regards MfG Robert Schetterer From robert at schetterer.org Sun Jul 22 23:33:51 2012 From: robert at schetterer.org (Robert Schetterer) Date: Sun, 22 Jul 2012 22:33:51 +0200 Subject: [Dovecot] Upgrade Problem from 2.0.18 to 2.1.7 In-Reply-To: <20120722152311.GA11986@daniel.localdomain> References: <500AC4F5.50209@gmx.de> <20120721210402.GA29891@daniel.localdomain> <500C1249.6050702@gmx.de> <20120722152311.GA11986@daniel.localdomain> Message-ID: <500C63AF.3040406@schetterer.org> Am 22.07.2012 17:23, schrieb Daniel Parthey: > Michael Domann wrote: >> Can i prevent dovecot from making new dirs? >> Because i have one draft and one entw?rfe dir, also one trash and >> one papierkorb. one german, one english. the dirs are always created >> new, after i delete them. > > Most probably, your mail clients are responsible for creating > the folders via IMAP. Dovecot only performs the commands from > your IMAP client, so this is rather a client issue. > > You will need to tell *all* your mail user agents and your > mobile phone to use the same folders for the same purpose. > > Regards > Daniel you may "advice" mailclients if they support it i.e http://dovecot.org/list/dovecot/2011-December/062327.html feature called IMAP SPECIAL-USE > -- Best Regards MfG Robert Schetterer From voytek at sbt.net.au Sun Jul 22 23:35:13 2012 From: voytek at sbt.net.au (Voytek Eymont) Date: Mon, 23 Jul 2012 06:35:13 +1000 Subject: [Dovecot] ot: execute a script via email? In-Reply-To: References: <3c0e8e3f-e68a-4354-89d6-0a2233f0809c@email.android.com> Message-ID: On Fri, July 13, 2012 9:28 pm, Ken Anderson wrote: > If you don't have root, you are probably going to be restricted from thanks for all the suggestions, for now I've settled on 'ostiary'; I'll look at the other suggestions later, thanks again -- Voytek From yeupou at gnu.org Mon Jul 23 04:37:01 2012 From: yeupou at gnu.org (Mathieu Roy) Date: Mon, 23 Jul 2012 03:37:01 +0200 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: <48EA2C62-814F-43CC-8D9A-FC8CD28E41A9@iki.fi> References: <201207221346.57165@bender.stalag13.ici> <48EA2C62-814F-43CC-8D9A-FC8CD28E41A9@iki.fi> Message-ID: <201207230337.01795@bender.stalag13.ici> Le dimanche 22 juillet 2012, Timo Sirainen a ?crit : > On 22.7.2012, at 14.46, Mathieu Roy wrote: > > > Hello, > > > > I was using dovecot 1.2.x via xinetd with a setup like http://wiki.dovecot.org/InetdInstall > > Since I upgraded to Debian Wheezy, shipping dovecot 2.x, it no longer works. At best, I end up with stuff like > > > > Error: net_connect_unix(anvil) failed: No such file or directory > > Fatal: Couldn't connect to anvil > > > > Any clues? > > Regards, > > > > (add me in Cc in replies, I'm not suscribed to the list) > > Doesn't work anymore. No plans to make it work anymore. Way too much trouble. Is there any way to make dovecot aware of hosts.deny and hosts.allow? Thanks for your reply, Regards, -- Mathieu Roy From tss at iki.fi Mon Jul 23 11:45:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Mon, 23 Jul 2012 11:45:20 +0300 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: <201207230337.01795@bender.stalag13.ici> References: <201207221346.57165@bender.stalag13.ici> <48EA2C62-814F-43CC-8D9A-FC8CD28E41A9@iki.fi> <201207230337.01795@bender.stalag13.ici> Message-ID: On 23.7.2012, at 4.37, Mathieu Roy wrote: >>> I was using dovecot 1.2.x via xinetd with a setup like http://wiki.dovecot.org/InetdInstall >>> Since I upgraded to Debian Wheezy, shipping dovecot 2.x, it no longer works. At best, I end up with stuff like >>> >> Doesn't work anymore. No plans to make it work anymore. Way too much trouble. > > Is there any way to make dovecot aware of hosts.deny and hosts.allow? If you've compiled with tcpwrappers, you can do: login_access_sockets = tcpwrap From fy at 5dshu.com Mon Jul 23 11:51:33 2012 From: fy at 5dshu.com (F.Y) Date: Mon, 23 Jul 2012 16:51:33 +0800 Subject: [Dovecot] about .dovecot.sieve file can't found for directory Message-ID: Hi all : i used roundcube webmail and enable filter(managesieve) . i create some rules ,and the rule save in maildir . ok ..see my dovecot config: # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.5 ext3 log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_uid: vuser mail_gid: vgroup mail_location: maildir:/home/data/domains/%d/%n/Maildir mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve managesieve_logout_format(default): bytes=%i/%o managesieve_logout_format(imap): bytes=%i/%o managesieve_logout_format(pop3): bytes=%i/%o managesieve_logout_format(managesieve): bytes(in=%i/out=%o) lda: postmaster_address: fy at 5dshu.com mail_plugins: sieve mail_plugin_dir: /usr/lib/dovecot/modules/lda auth_socket_path: /var/run/dovecot/auth-master auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vuser group: vgroup plugin: sieve_dir: /home/data/domains/%d/%n/Maildir/sieve sieve: /home/data/domains/%d/%n/Maildir/.dovecot.sieve I create some rules, it can save for Maildir/sieve ,but can't found .dovecot.sieve on Maildir.. i trying for google and baidu search engine,can't found any help . From CMarcus at Media-Brokers.com Mon Jul 23 12:06:25 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Mon, 23 Jul 2012 05:06:25 -0400 Subject: [Dovecot] maildir_copy_with_hardlinks on v.2.0.19 In-Reply-To: References: Message-ID: <500D1411.4030808@Media-Brokers.com> On 2012-07-22 7:59 AM, mailing list subscriber wrote: > I'm trying to get the so-called "single instance store" (I think cyrus > has got the name for the first time) with dovecot --version = 2.0.19 > binary package installed from ubuntu 12.04 lts official repo. For advanced/newest features, you would be well advised to use the most current stable version, which is currently 2.1.8... > I have checked that "maildir_copy_with_hardlinks" is enabled ("dovecot > -a|grep hard" shows "yes") then I have installed and enabled the lmtp > component of dovecot. The configuration "dovecot -n" is pasted here: > http://paste.lug.ro/131180 Please don't do that - just paste the output in the email body, it is much easier to read/respond and try to help you. -- Best regards, Charles From stephan at rename-it.nl Mon Jul 23 12:21:47 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Mon, 23 Jul 2012 11:21:47 +0200 Subject: [Dovecot] about .dovecot.sieve file can't found for directory In-Reply-To: References: Message-ID: <500D17AB.3050407@rename-it.nl> Op 7/23/2012 10:51 AM, F.Y schreef: > Hi all : > i used roundcube webmail and enable filter(managesieve) . i create > some rules ,and the rule save in maildir . ok ..see my dovecot config: > > mail_location: maildir:/home/data/domains/%d/%n/Maildir > > plugin: > sieve_dir: /home/data/domains/%d/%n/Maildir/sieve > sieve: /home/data/domains/%d/%n/Maildir/.dovecot.sieve > > I create some rules, it can save for Maildir/sieve ,but can't found > .dovecot.sieve on Maildir.. i trying for google and baidu search > engine,can't found any help . Never put sieve and sieve_dir inside your mail_location, especially when it is a Maildir. So, this should solve it for you: plugin: sieve_dir: /home/data/domains/%d/%n/sieve sieve: /home/data/domains/%d/%n/.dovecot.sieve Also make sure you have a sensible home directory set (e.g. /home/data/domains/%d/%n), since some aspects of Sieve use that for storing state information. Other options exist of course. Regards, Stephan. From yeupou at gnu.org Mon Jul 23 13:54:46 2012 From: yeupou at gnu.org (Mathieu Roy) Date: Mon, 23 Jul 2012 12:54:46 +0200 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: References: <201207221346.57165@bender.stalag13.ici> <201207230337.01795@bender.stalag13.ici> Message-ID: <201207231254.46758@bender.stalag13.ici> Le lundi 23 juillet 2012, Timo Sirainen a ?crit : > If you've compiled with tcpwrappers, you can do: > > login_access_sockets = tcpwrap I use debian stock version, I'll check that and recompile if need be. Thanks a lot, -- Mathieu Roy From stan at hardwarefreak.com Mon Jul 23 14:23:14 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 23 Jul 2012 06:23:14 -0500 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: <201207231254.46758@bender.stalag13.ici> References: <201207221346.57165@bender.stalag13.ici> <201207230337.01795@bender.stalag13.ici> <201207231254.46758@bender.stalag13.ici> Message-ID: <500D3422.7090201@hardwarefreak.com> On 7/23/2012 5:54 AM, Mathieu Roy wrote: > Le lundi 23 juillet 2012, Timo Sirainen a ?crit : >> If you've compiled with tcpwrappers, you can do: >> >> login_access_sockets = tcpwrap > > I use debian stock version, I'll check that and recompile if need be. Debian stable has Dovecot 1.2.15, and I don't see 2.x in backports. Yet you're discussing 2.x. You must be using testing which has 2.1.7. In which case using the word "stock" isn't quite appropriate, as it implies the stable branch. You should have explicitly stated which branch you're using to avoid confusion. -- Stan From e-frog at gmx.de Mon Jul 23 14:52:12 2012 From: e-frog at gmx.de (e-frog) Date: Mon, 23 Jul 2012 13:52:12 +0200 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: <500D3422.7090201@hardwarefreak.com> References: <201207221346.57165@bender.stalag13.ici> <201207230337.01795@bender.stalag13.ici> <201207231254.46758@bender.stalag13.ici> <500D3422.7090201@hardwarefreak.com> Message-ID: <500D3AEC.2040408@gmx.de> On 23.07.2012 13:23, wrote Stan Hoeppner: > On 7/23/2012 5:54 AM, Mathieu Roy wrote: >> Le lundi 23 juillet 2012, Timo Sirainen a ?crit : >>> If you've compiled with tcpwrappers, you can do: >>> >>> login_access_sockets = tcpwrap >> >> I use debian stock version, I'll check that and recompile if need be. > > Debian stable has Dovecot 1.2.15, and I don't see 2.x in backports. Yet > you're discussing 2.x. You must be using testing which has 2.1.7. In > which case using the word "stock" isn't quite appropriate, as it implies > the stable branch. You should have explicitly stated which branch > you're using to avoid confusion. > 2.1.7 is indeed in backports http://packages.debian.org/search?keywords=dovecot&searchon=names§ion=all&suite=squeeze-backports From stan at hardwarefreak.com Mon Jul 23 15:11:25 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Mon, 23 Jul 2012 07:11:25 -0500 Subject: [Dovecot] dovecot 2.x via xinetd In-Reply-To: <500D3AEC.2040408@gmx.de> References: <201207221346.57165@bender.stalag13.ici> <201207230337.01795@bender.stalag13.ici> <201207231254.46758@bender.stalag13.ici> <500D3422.7090201@hardwarefreak.com> <500D3AEC.2040408@gmx.de> Message-ID: <500D3F6D.2050208@hardwarefreak.com> On 7/23/2012 6:52 AM, e-frog wrote: > On 23.07.2012 13:23, wrote Stan Hoeppner: >> On 7/23/2012 5:54 AM, Mathieu Roy wrote: >>> Le lundi 23 juillet 2012, Timo Sirainen a ?crit : >>>> If you've compiled with tcpwrappers, you can do: >>>> >>>> login_access_sockets = tcpwrap >>> >>> I use debian stock version, I'll check that and recompile if need be. >> >> Debian stable has Dovecot 1.2.15, and I don't see 2.x in backports. Yet >> you're discussing 2.x. You must be using testing which has 2.1.7. In >> which case using the word "stock" isn't quite appropriate, as it implies >> the stable branch. You should have explicitly stated which branch >> you're using to avoid confusion. >> > > 2.1.7 is indeed in backports > > http://packages.debian.org/search?keywords=dovecot&searchon=names§ion=all&suite=squeeze-backports Good catch e-frog. I stand (partially) corrected. Indeed it is in backports. I hadn't done an aptitude update for a while, though it hasn't been more than a month, so this is very new. Debian doesn't provide a package inclusion date... I say "partially" because Backports packages don't really fall under the term "stock" either. -- Stan From campbell at cnpapers.com Mon Jul 23 18:31:38 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 23 Jul 2012 11:31:38 -0400 Subject: [Dovecot] Couple of questions about the logs Message-ID: <500D6E5A.1040202@cnpapers.com> The log entries for imap disconnection shows a "bytes = x/y" format where the x equals bytes sent from client and y equals bytes received from client. Can someone explain that a little better to me, please? In an imap account, does this "y" represent the size of the header information being returned to the client or the amount of data of the email mailbox? Also, is there a way to tell if an email has been deleted (or marked for deletion) when using an imap client? Here's the situation: I've got a user who uses imap when using his phone and webmail. When in the office, he uses a pop3 client on his desktop. He indicates he's missing some emails from the weekend when first turning on his desktop. I know of no way to discover if he's deleted and then purged email from his mailbox using imap clients. I've checked the logs and he does not access his mailbox simultaneously from any of the 3 clients. I can find 31 emails he received since his last pop login on Friday until his first pop login on Monday. The pop login indicates he received only 27 emails during the pop retrieval. I'm a little lost here. Any help would be appreciated in interpreting the logs or suggesting how this might have happened. thanks steve campbell From e-frog at gmx.de Mon Jul 23 18:54:46 2012 From: e-frog at gmx.de (e-frog) Date: Mon, 23 Jul 2012 17:54:46 +0200 Subject: [Dovecot] Couple of questions about the logs In-Reply-To: <500D6E5A.1040202@cnpapers.com> References: <500D6E5A.1040202@cnpapers.com> Message-ID: <500D73C6.9040408@gmx.de> On 23.07.2012 17:31, wrote Steve Campbell: > The log entries for imap disconnection shows a "bytes = x/y" format > where the x equals bytes sent from client and y equals bytes received > from client. > > Can someone explain that a little better to me, please? In an imap > account, does this "y" represent the size of the header information > being returned to the client or the amount of data of the email mailbox? > > Also, is there a way to tell if an email has been deleted (or marked for > deletion) when using an imap client? > > Here's the situation: > > I've got a user who uses imap when using his phone and webmail. When in > the office, he uses a pop3 client on his desktop. He indicates he's > missing some emails from the weekend when first turning on his desktop. > I know of no way to discover if he's deleted and then purged email from > his mailbox using imap clients. I've checked the logs and he does not > access his mailbox simultaneously from any of the 3 clients. > > I can find 31 emails he received since his last pop login on Friday > until his first pop login on Monday. The pop login indicates he received > only 27 emails during the pop retrieval. > > I'm a little lost here. Any help would be appreciated in interpreting > the logs or suggesting how this might have happened. > > thanks > > steve campbell This probably doesn't help with the current case but for the future you could enable mail_log plugin to log several user events: http://wiki2.dovecot.org/Plugins/MailLog From campbell at cnpapers.com Mon Jul 23 19:07:21 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 23 Jul 2012 12:07:21 -0400 Subject: [Dovecot] Couple of questions about the logs In-Reply-To: <500D73C6.9040408@gmx.de> References: <500D6E5A.1040202@cnpapers.com> <500D73C6.9040408@gmx.de> Message-ID: <500D76B9.40205@cnpapers.com> On 7/23/2012 11:54 AM, e-frog wrote: > On 23.07.2012 17:31, wrote Steve Campbell: >> The log entries for imap disconnection shows a "bytes = x/y" format >> where the x equals bytes sent from client and y equals bytes received >> from client. >> >> Can someone explain that a little better to me, please? In an imap >> account, does this "y" represent the size of the header information >> being returned to the client or the amount of data of the email mailbox? >> >> Also, is there a way to tell if an email has been deleted (or marked for >> deletion) when using an imap client? >> >> Here's the situation: >> >> I've got a user who uses imap when using his phone and webmail. When in >> the office, he uses a pop3 client on his desktop. He indicates he's >> missing some emails from the weekend when first turning on his desktop. >> I know of no way to discover if he's deleted and then purged email from >> his mailbox using imap clients. I've checked the logs and he does not >> access his mailbox simultaneously from any of the 3 clients. >> >> I can find 31 emails he received since his last pop login on Friday >> until his first pop login on Monday. The pop login indicates he received >> only 27 emails during the pop retrieval. >> >> I'm a little lost here. Any help would be appreciated in interpreting >> the logs or suggesting how this might have happened. >> >> thanks >> >> steve campbell > > This probably doesn't help with the current case but for the future > you could enable mail_log plugin to log several user events: > > http://wiki2.dovecot.org/Plugins/MailLog thanks very much. That looks great. Is there any particular file I should place the particular lines? steve From e-frog at gmx.de Mon Jul 23 19:23:26 2012 From: e-frog at gmx.de (e-frog) Date: Mon, 23 Jul 2012 18:23:26 +0200 Subject: [Dovecot] Couple of questions about the logs In-Reply-To: <500D76B9.40205@cnpapers.com> References: <500D6E5A.1040202@cnpapers.com> <500D73C6.9040408@gmx.de> <500D76B9.40205@cnpapers.com> Message-ID: <500D7A7E.4010504@gmx.de> On 23.07.2012 18:07, wrote Steve Campbell: > > On 7/23/2012 11:54 AM, e-frog wrote: >> On 23.07.2012 17:31, wrote Steve Campbell: >>> The log entries for imap disconnection shows a "bytes = x/y" format >>> where the x equals bytes sent from client and y equals bytes received >>> from client. >>> >>> Can someone explain that a little better to me, please? In an imap >>> account, does this "y" represent the size of the header information >>> being returned to the client or the amount of data of the email mailbox? >>> >>> Also, is there a way to tell if an email has been deleted (or marked for >>> deletion) when using an imap client? >>> >>> Here's the situation: >>> >>> I've got a user who uses imap when using his phone and webmail. When in >>> the office, he uses a pop3 client on his desktop. He indicates he's >>> missing some emails from the weekend when first turning on his desktop. >>> I know of no way to discover if he's deleted and then purged email from >>> his mailbox using imap clients. I've checked the logs and he does not >>> access his mailbox simultaneously from any of the 3 clients. >>> >>> I can find 31 emails he received since his last pop login on Friday >>> until his first pop login on Monday. The pop login indicates he received >>> only 27 emails during the pop retrieval. >>> >>> I'm a little lost here. Any help would be appreciated in interpreting >>> the logs or suggesting how this might have happened. >>> >>> thanks >>> >>> steve campbell >> >> This probably doesn't help with the current case but for the future >> you could enable mail_log plugin to log several user events: >> >> http://wiki2.dovecot.org/Plugins/MailLog > > thanks very much. That looks great. > > Is there any particular file I should place the particular lines? > > steve For dovecot 2.x.x conf.d/20-imap.conf: add mail_log and notify to mail_plugins mail_plugins = $mail_plugins mail_log notify conf.d/10-logging.conf: There is already a template, just uncomment and modify as needed. # mail_log plugin provides more event logging for mail processes. plugin { # Events to log. Also available: flag_change append #mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Available fields: uid, box, msgid, from, subject, size, vsize, flags # size and vsize are available only for expunge and copy events. #mail_log_fields = uid box msgid size } From campbell at cnpapers.com Mon Jul 23 19:44:30 2012 From: campbell at cnpapers.com (Steve Campbell) Date: Mon, 23 Jul 2012 12:44:30 -0400 Subject: [Dovecot] Couple of questions about the logs In-Reply-To: <500D7A7E.4010504@gmx.de> References: <500D6E5A.1040202@cnpapers.com> <500D73C6.9040408@gmx.de> <500D76B9.40205@cnpapers.com> <500D7A7E.4010504@gmx.de> Message-ID: <500D7F6E.1010203@cnpapers.com> On 7/23/2012 12:23 PM, e-frog wrote: > On 23.07.2012 18:07, wrote Steve Campbell: >> >> On 7/23/2012 11:54 AM, e-frog wrote: >>> On 23.07.2012 17:31, wrote Steve Campbell: >>>> The log entries for imap disconnection shows a "bytes = x/y" format >>>> where the x equals bytes sent from client and y equals bytes received >>>> from client. >>>> >>>> Can someone explain that a little better to me, please? In an imap >>>> account, does this "y" represent the size of the header information >>>> being returned to the client or the amount of data of the email >>>> mailbox? >>>> >>>> Also, is there a way to tell if an email has been deleted (or >>>> marked for >>>> deletion) when using an imap client? >>>> >>>> Here's the situation: >>>> >>>> I've got a user who uses imap when using his phone and webmail. >>>> When in >>>> the office, he uses a pop3 client on his desktop. He indicates he's >>>> missing some emails from the weekend when first turning on his >>>> desktop. >>>> I know of no way to discover if he's deleted and then purged email >>>> from >>>> his mailbox using imap clients. I've checked the logs and he does not >>>> access his mailbox simultaneously from any of the 3 clients. >>>> >>>> I can find 31 emails he received since his last pop login on Friday >>>> until his first pop login on Monday. The pop login indicates he >>>> received >>>> only 27 emails during the pop retrieval. >>>> >>>> I'm a little lost here. Any help would be appreciated in interpreting >>>> the logs or suggesting how this might have happened. >>>> >>>> thanks >>>> >>>> steve campbell >>> >>> This probably doesn't help with the current case but for the future >>> you could enable mail_log plugin to log several user events: >>> >>> http://wiki2.dovecot.org/Plugins/MailLog >> >> thanks very much. That looks great. >> >> Is there any particular file I should place the particular lines? >> >> steve > > For dovecot 2.x.x > > conf.d/20-imap.conf: add mail_log and notify to mail_plugins > > mail_plugins = $mail_plugins mail_log notify > > > conf.d/10-logging.conf: There is already a template, just uncomment > and modify as needed. > > # mail_log plugin provides more event logging for mail processes. > plugin { > # Events to log. Also available: flag_change append > #mail_log_events = delete undelete expunge copy mailbox_delete > mailbox_rename > # Available fields: uid, box, msgid, from, subject, size, vsize, flags > # size and vsize are available only for expunge and copy events. > #mail_log_fields = uid box msgid size > } I found the 10-logging.conf file, but wasn't sure where to put the "mail_plugins" line. Thanks very much for the help. My setup is as follows: # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.el6.x86_64 x86_64 CentOS release 6.2 (Final) disable_plaintext_auth = no listen = * mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mbox_write_locks = fcntl namespace { hidden = yes inbox = yes list = yes location = mbox:~/mail:INBOX=/var/spool/mail/%u prefix = separator = / type = private } namespace { hidden = yes list = no location = mbox:~/mail:INBOX=/var/spool/mail/%u prefix = "#mbox/" separator = / type = private } namespace { hidden = yes list = no location = prefix = mail/ separator = / type = private } namespace { hidden = yes list = no location = mbox:~/mail:INBOX=/var/spool/mail/%u prefix = ~/mail/ separator = / type = private } namespace { hidden = yes list = no location = prefix = ~%u/mail/ separator = / type = private } passdb { driver = pam } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = imap pop3 ssl_cert = References: Message-ID: On Sun, Jul 22, 2012 at 2:59 PM, mailing list subscriber wrote: > Hi, > > I'm trying to get the so-called "single instance store" (I think cyrus > has got the name for the first time) with dovecot --version = 2.0.19 > binary package installed from ubuntu 12.04 lts official repo. As requested, here is my update. As you can see I am running now the latest release however emails delivered through lmtp gets split into different files instead of expected hardlinked files. Please pay attention as I'm commenting in-between different pasted outputs: # 2.1.8 (30b0d6b1c581): /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-26-generic x86_64 Ubuntu 12.04 LTS auth_username_format = %Ln auth_verbose = yes auth_verbose_passwords = plain mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = prefix = } passdb { driver = pam } plugin { autocreate = Inbox autocreate2 = Sent autocreate3 = Drafts autocreate4 = Spam autocreate5 = Trash autosubscribe = Inbox autosubscribe2 = Sent autosubscribe3 = Drafts autosubscribe4 = Spam autosubscribe5 = Trash mail_log_events = delete expunge mailbox_delete mailbox_rename mail_log_fields = uid box msgid size sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve lmtp service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = was automatically rejected:%n%r } for your pleasure, here is the transaction (note the same queue id and a single lmtp session for all three recipients) Jul 23 21:47:32 imap postfix/qmgr[27463]: 6746C27C687: from=, size=532, nrcpt=3 (queue active) Jul 23 21:47:32 imap dovecot: lmtp(40609): Connect from local Jul 23 21:47:32 imap dovecot: lmtp(40609, anotheruser): pnuDHkScDVChngAA7nOI2A: msgid=<20120723184732.6746C27C687 at imap.mydomain.ro>: saved mail to INBOX Jul 23 21:47:32 imap postfix/lmtp[40608]: 6746C27C687: to=, relay=imap.mydomain.ro[private/dovecot-lmtp], delay=0.26, delays=0.15/0.01/0/0.1, dsn=2.0.0, status=sent (250 2.0.0 pnuDHkScDVChngAA7nOI2A Saved) Jul 23 21:47:32 imap dovecot: lmtp(40609, firstuser): pnuDHkScDVChngAA7nOI2A: msgid=<20120723184732.6746C27C687 at imap.mydomain.ro>: saved mail to INBOX Jul 23 21:47:32 imap postfix/lmtp[40608]: 6746C27C687: to=, relay=imap.mydomain.ro[private/dovecot-lmtp], delay=0.37, delays=0.15/0.01/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 pnuDHkScDVChngAA7nOI2A Saved) Jul 23 21:47:32 imap dovecot: lmtp(40609, firstuser): pnuDHkScDVChngAA7nOI2A: msgid=<20120723184732.6746C27C687 at imap.mydomain.ro>: saved mail to INBOX Jul 23 21:47:32 imap postfix/lmtp[40608]: 6746C27C687: to=, relay=imap.mydomain.ro[private/dovecot-lmtp], delay=0.44, delays=0.15/0.01/0/0.28, dsn=2.0.0, status=sent (250 2.0.0 pnuDHkScDVChngAA7nOI2A Saved) Jul 23 21:47:32 imap dovecot: lmtp(40609): Disconnect from local: Client quit (in reset) Jul 23 21:47:32 imap postfix/qmgr[27463]: 6746C27C687: removed for whoever wants to blame postfix 2.9.3 without reason, here is the postconf -n alias_database = hash:/etc/aliases alias_maps = hash:/etc/aliases biff = no broken_sasl_auth_clients = yes config_directory = /etc/postfix home_mailbox = Maildir/ html_directory = /usr/share/doc/postfix/html inet_interfaces = all mailbox_command = /usr/lib/dovecot/deliver -c /etc/dovecot/conf.d/01-mail-stack-delivery.conf -m "${EXTENSION}" mailbox_size_limit = 0 mydestination = imap.mydomain.ro, localhost.mydomain.ro, localhost myhostname = imap.mydomain.ro mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 readme_directory = /usr/share/doc/postfix recipient_delimiter = + relayhost = smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_use_tls = yes smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) smtpd_recipient_restrictions = reject_unknown_sender_domain, reject_unknown_recipient_domain, reject_unauth_pipelining, permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination smtpd_sasl_auth_enable = yes smtpd_sasl_authenticated_header = yes smtpd_sasl_local_domain = $myhostname smtpd_sasl_path = private/dovecot-auth smtpd_sasl_security_options = noanonymous smtpd_sasl_type = dovecot smtpd_sender_restrictions = reject_unknown_sender_domain smtpd_tls_auth_only = yes smtpd_tls_cert_file = /etc/ssl/certs/ssl-mail.pem smtpd_tls_key_file = /etc/ssl/private/ssl-mail.key smtpd_tls_mandatory_ciphers = medium smtpd_tls_mandatory_protocols = SSLv3, TLSv1 smtpd_tls_received_header = yes smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtpd_use_tls = yes tls_random_source = dev:/dev/urandom virtual_alias_maps = hash:/etc/postfix/valid_recipients_alias virtual_mailbox_domains = mydomain.ro, anothermydomain.ro virtual_mailbox_maps = $virtual_alias_maps virtual_transport = lmtp:unix:private/dovecot-lmtp and the dialogue between postfix lmtp client and dovecot lmtp server: Process 40677 attached (waiting for parent) Process 40677 resumed (parent 40562 ready) [pid 40676] write(4, "VERSION\tconfig\t2\t0\nREQ\tmodule=lmtp\tservice=lmtp\n", 48) = 48 [pid 40676] read(4, [Process 40678 attached Process 40677 suspended [pid 40678] write(1, "SIEVE: fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave, NOTIFY: mailto\n", 208) = 208 Process 40677 resumed Process 40678 detached [pid 40677] --- SIGCHLD (Child exited) @ 0 (0) --- [pid 40677] read(11, "SIEVE: fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave, NOTIFY: mailto\n", 4096) = 208 [pid 40677] read(11, "", 3888) = 0 [pid 40677] read(21, "", 1024) = 0 [pid 40677] read(21, "", 1024) = 0 [pid 40677] read(10, "# /etc/nsswitch.conf\n#\n# Example configuration of GNU Name Service Switch functionality.\n# If you have the `glibc-doc-reference' and `info' packages installed, try:\n# `info libc \"Name Service Switch\"' for information about this file.\n\npasswd: compat\ngroup: compat\nshadow: compat\n\nhosts: files dns\nnetworks: files\n\nprotocols: db files\nservices: db files\nethers: db files\nrpc: db files\n\nnetgroup: nis\n", 4096) = 475 [pid 40677] read(10, "", 4096) = 0 [pid 40677] read(10, "VERSION\tconfig\t2\t0\nREQ\tmodule=lmtp\tservice=lmtp\n", 1024) = 48 [pid 40677] write(10, "\nnamespace=inbox\nnamespace/inbox/name=inbox\nnamespace/inbox/inbox=yes\nplugin=\nplugin/sieve=~/.dovecot.sieve\nplugin/sieve_dir=~/sieve\nplugin/autocreate=Inbox\nplugin/autocreate2=Sent\nplugin/autocreate3=Drafts\nplugin/autocreate4=Spam\nplugin/autocreate5=Trash\nplugin/autosubscribe=Inbox\nplugin/autosubscribe2=Sent\nplugin/autosubscribe3=Drafts\nplugin/autosubscribe4=Spam\nplugin/autosubscribe5=Trash\nplugin/mail_log_events=delete expunge mailbox_delete mailbox_rename\nplugin/mail_log_fields=uid box msgid size\nmail_location=maildir:~/Maildir\nmaildir_copy_with_hardlinks=yes\nverbose_proctitle=yes\n\n", 591) = 591 [pid 40676] <... read resumed> "\nnamespace=inbox\nnamespace/inbox/name=inbox\nnamespace/inbox/inbox=yes\nplugin=\nplugin/sieve=~/.dovecot.sieve\nplugin/sieve_dir=~/sieve\nplugin/autocreate=Inbox\nplugin/autocreate2=Sent\nplugin/autocreate3=Drafts\nplugin/autocreate4=Spam\nplugin/autocreate5=Trash\nplugin/autosubscribe=Inbox\nplugin/autosubscribe2=Sent\nplugin/autosubscribe3=Drafts\nplugin/autosubscribe4=Spam\nplugin/autosubscribe5=Trash\nplugin/mail_log_events=delete expunge mailbox_delete mailbox_rename\nplugin/mail_log_fields=uid box msgid size\nmail_location=maildir:~/Maildir\nmaildir_copy_with_hardlinks=yes\nverbose_proctitle=yes\n\n", 8192) = 591 [pid 40676] write(5, "\344\236\0\0\7\0\0\0\1\0\0\0", 12) = 12 [pid 40562] read(128, "\344\236\0\0\7\0\0\0\1\0\0\0", 1020) = 12 [pid 40676] write(5, "\344\236\0\0\7\0\0\0\0\0\0\0", 12) = 12 [pid 40562] read(128, "\344\236\0\0\7\0\0\0\0\0\0\0", 1020) = 12 [pid 40676] write(4, "VERSION\tconfig\t2\t0\nREQ\tmodule=lmtp\tservice=lmtp\tuser=\n", 54) = 54 [pid 40677] read(10, "VERSION\tconfig\t2\t0\nREQ\tmodule=lmtp\tservice=lmtp\tuser=\n", 976) = 54 [pid 40676] read(4, [pid 40677] write(10, "\nnamespace=inbox\nnamespace/inbox/name=inbox\nnamespace/inbox/inbox=yes\nplugin=\nplugin/sieve=~/.dovecot.sieve\nplugin/sieve_dir=~/sieve\nplugin/autocreate=Inbox\nplugin/autocreate2=Sent\nplugin/autocreate3=Drafts\nplugin/autocreate4=Spam\nplugin/autocreate5=Trash\nplugin/autosubscribe=Inbox\nplugin/autosubscribe2=Sent\nplugin/autosubscribe3=Drafts\nplugin/autosubscribe4=Spam\nplugin/autosubscribe5=Trash\nplugin/mail_log_events=delete expunge mailbox_delete mailbox_rename\nplugin/mail_log_fields=uid box msgid size\nmail_location=maildir:~/Maildir\nmaildir_copy_with_hardlinks=yes\nverbose_proctitle=yes\n\n", 591) = 591 [pid 40676] <... read resumed> "\nnamespace=inbox\nnamespace/inbox/name=inbox\nnamespace/inbox/inbox=yes\nplugin=\nplugin/sieve=~/.dovecot.sieve\nplugin/sieve_dir=~/sieve\nplugin/autocreate=Inbox\nplugin/autocreate2=Sent\nplugin/autocreate3=Drafts\nplugin/autocreate4=Spam\nplugin/autocreate5=Trash\nplugin/autosubscribe=Inbox\nplugin/autosubscribe2=Sent\nplugin/autosubscribe3=Drafts\nplugin/autosubscribe4=Spam\nplugin/autosubscribe5=Trash\nplugin/mail_log_events=delete expunge mailbox_delete mailbox_rename\nplugin/mail_log_fields=uid box msgid size\nmail_location=maildir:~/Maildir\nmaildir_copy_with_hardlinks=yes\nverbose_proctitle=yes\n\n", 8192) = 591 [pid 40676] read(13, "# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)\n# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN\nnameserver 192.168.17.1\nsearch mydomain.ro\n", 4096) = 202 [pid 40676] read(13, "", 4096) = 0 [pid 40676] read(13, "# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)\n# DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN\nnameserver 192.168.17.1\nsearch mydomain.ro\n", 4096) = 202 [pid 40676] read(13, "", 4096) = 0 [pid 40676] read(13, "# /etc/nsswitch.conf\n#\n# Example configuration of GNU Name Service Switch functionality.\n# If you have the `glibc-doc-reference' and `info' packages installed, try:\n# `info libc \"Name Service Switch\"' for information about this file.\n\npasswd: compat\ngroup: compat\nshadow: compat\n\nhosts: files dns\nnetworks: files\n\nprotocols: db files\nservices: db files\nethers: db files\nrpc: db files\n\nnetgroup: nis\n", 4096) = 475 [pid 40676] read(13, "", 4096) = 0 [pid 40676] read(13, "# The \"order\" line is only used by old versions of the C library.\norder hosts,bind\nmulti on\n", 4096) = 92 [pid 40676] read(13, "", 4096) = 0 [pid 40676] read(13, "127.0.0.1\tlocalhost\n192.168.17.71\timap.mydomain.ro\timap\n\n# The following lines are desirable for IPv6 capable hosts\n::1 ip6-localhost ip6-loopback\nfe00::0 ip6-localnet\nff00::0 ip6-mcastprefix\nff02::1 ip6-allnodes\nff02::2 ip6-allrouters\n", 4096) = 248 [pid 40676] read(13, "", 4096) = 0 [pid 40676] write(12, "220 imap.mydomain.ro Dovecot ready.\r\n", 45) = 45 [pid 40676] write(2, "\1\01040676 prefix=lmtp(40676): \n", 29) = 29 [pid 40676] write(2, "\1\00240676 Connect from local\n", 27) = 27 [pid 40676] read(12, "LHLO imap.mydomain.ro\r\n", 4096) = 31 [pid 40676] write(12, "250-imap.mydomain.ro\r\n250-8BITMIME\r\n250-ENHANCEDSTATUSCODES\r\n250 PIPELINING\r\n", 85) = 85 [pid 40676] read(12, "MAIL FROM:\r\nRCPT TO:\r\nRCPT TO:\r\nRCPT TO:\r\nDATA\r\n", 4065) = 163 Process 40679 attached [pid 40676] write(13, "VERSION\t1\t0\nUSER\t1\tanotheruser at mydomain.ro\tservice=lmtp\n", 64) = 64 [pid 40679] read(4, "+\267\361\217", 4) = 4 [pid 40679] write(13, "VERSION\tconfig\t2\t0\nREQ\tmodule=auth\n", 35) = 35 [pid 40679] read(13, [pid 40677] read(11, "VERSION\tconfig\t2\t0\nREQ\tmodule=auth\n", 1024) = 35 [pid 40677] write(11, "\nverbose_proctitle=yes\nauth_mechanisms=plain\nauth_username_chars=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@\nauth_username_format=%Ln\nauth_verbose=yes\nauth_verbose_passwords=plain\npassdb=0\npassdb/0/driver=pam\nuserdb=0\nuserdb/0/driver=passwd\n\n", 266 [pid 40679] <... read resumed> "\nverbose_proctitle=yes\nauth_mechanisms=plain\nauth_username_chars=abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@\nauth_username_format=%Ln\nauth_verbose=yes\nauth_verbose_passwords=plain\npassdb=0\npassdb/0/driver=pam\nuserdb=0\nuserdb/0/driver=passwd\n\n", 8192) = 266 [pid 40677] <... write resumed> ) = 266 [pid 40677] read(11, "", 989) = 0 [pid 40562] read(7, "t\236\0\0\1\0\0\0\347\3\0\0", 1020) = 12 [pid 40679] write(13, "VERSION\tanvil\t1\t0\n", 18) = 18 [pid 40679] write(5, "\347\236\0\0\t\0\0\0\350\3\0\0", 12) = 12 [pid 40562] read(151, "\347\236\0\0\t\0\0\0\350\3\0\0", 1020) = 12 [pid 40679] write(17, "VERSION\t1\t1\nSPID\t40679\n", 23) = 23 [pid 40676] read(13, "VERSION\t1\t1\nSPID\t40679\n", 8192) = 23 [pid 40679] read(17, "VERSION\t1\t0\nUSER\t1\tanotheruser at mydomain.ro\tservice=lmtp\n", 1024) = 64 [pid 40679] read(18, "# /etc/nsswitch.conf\n#\n# Example configuration of GNU Name Service Switch functionality.\n# If you have the `glibc-doc-reference' and `info' packages installed, try:\n# `info libc \"Name Service Switch\"' for information about this file.\n\npasswd: compat\ngroup: compat\nshadow: compat\n\nhosts: files dns\nnetworks: files\n\nprotocols: db files\nservices: db files\nethers: db files\nrpc: db files\n\nnetgroup: nis\n", 4096) = 475 [pid 40679] read(18, "", 4096) = 0 [pid 40679] write(17, "USER\t1\tanotheruser\tsystem_groups_user=anotheruser\tuid=1002\tgid=1002\thome=/home/anotheruser\n", 91) = 91 [pid 40676] read(13, "USER\t1\tanotheruser\tsystem_groups_user=anotheruser\tuid=1002\tgid=1002\thome=/home/anotheruser\n", 8169) = 91 [pid 40676] write(13, "USER\t2\tfirstuser at mydomain.ro\tservice=lmtp\n", 50) = 50 [pid 40679] read(17, "USER\t2\tfirstuser at mydomain.ro\tservice=lmtp\n", 960) = 50 [pid 40679] write(17, "USER\t2\tfirstuser\tsystem_groups_user=firstuser\tuid=1001\tgid=1001\thome=/home/firstuser\n", 85) = 85 [pid 40676] read(13, "USER\t2\tfirstuser\tsystem_groups_user=firstuser\tuid=1001\tgid=1001\thome=/home/firstuser\n", 8192) = 85 [pid 40676] write(13, "USER\t3\tfirstuser at anothermydomain.ro\tservice=lmtp\n", 39) = 39 [pid 40679] read(17, "USER\t3\tfirstuser at anothermydomain.ro\tservice=lmtp\n", 910) = 39 [pid 40679] write(17, "USER\t3\tfirstuser\tsystem_groups_user=firstuser\tuid=1001\tgid=1001\thome=/home/firstuser\n", 85) = 85 [pid 40676] read(13, "USER\t3\tfirstuser\tsystem_groups_user=firstuser\tuid=1001\tgid=1001\thome=/home/firstuser\n", 8192) = 85 [pid 40676] read(12, 0x18977e2, 3902) = -1 EAGAIN (Resource temporarily unavailable) [pid 40676] write(12, "250 2.1.0 OK\r\n250 2.1.5 OK\r\n250 2.1.5 OK\r\n250 2.1.5 OK\r\n354 OK\r\n", 64) = 64 [pid 40676] read(12, "Received: by imap.mydomain.ro (Postfix, from userid 0)\r\n\tid AE18F27C687; Mon, 23 Jul 2012 22:00:37 +0300 (EEST)\r\nDate: Mon, 23 Jul 2012 22:00:37 +0300\r\nTo: firstuser at anothermydomain.ro, firstuser at mydomain.ro\r\nCc: anotheruser at mydomain.ro\r\nSubject: test\r\nUser-Agent: Heirloom mailx 12.5 6/20/10\r\nMIME-Version: 1.0\r\nContent-Type: text/plain; charset=us-ascii\r\nContent-Transfer-Encoding: quoted-printable\r\nMessage-Id: <20120723190037.AE18F27C687 at imap.mydomain.ro>\r\nFrom: root at imap.mydomain.ro (root)\r\n\r\n=\r\n.\r\nQUIT\r\n", 3902) = 541 [pid 40676] read(14, "65536\n", 31) = 6 [pid 40676] write(16, "Return-Path: \nReceived: from imap.mydomain.ro\n\tby imap.mydomain.ro (Dovecot) with LMTP id /RqJNFWfDVDkngAA7nOI2A\n\t; Mon, 23 Jul 2012 22:00:37 +0300\nReceived: by imap.mydomain.ro (Postfix, from userid 0)\n\tid AE18F27C687; Mon, 23 Jul 2012 22:00:37 +0300 (EEST)\nDate: Mon, 23 Jul 2012 22:00:37 +0300\nTo: firstuser at anothermydomain.ro, firstuser at mydomain.ro\nCc: anotheruser at mydomain.ro\nSubject: test\nUser-Agent: Heirloom mailx 12.5 6/20/10\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\nMessage-Id: <20120723190037.AE18F27C687 at imap.mydomain.ro>\nFrom: root at imap.mydomain.ro (root)\n\n=\n", 713) = 713 [pid 40676] write(16, "40676:imap", 10) = 10 [pid 40676] write(15, "15 :1343070037.M903400P40676.imap,S=713,W=731\n", 46) = 46 [pid 40676] write(2, "\1\01040676 prefix=lmtp(40676, anotheruser): \n", 42) = 42 [pid 40676] write(2, "\1\00240676 /RqJNFWfDVDkngAA7nOI2A: msgid=<20120723190037.AE18F27C687 at imap.mydomain.ro>: saved mail to INBOX\n", 113) = 113 [pid 40676] write(16, "40676:imap", 10) = 10 [pid 40676] write(12, "250 2.0.0 /RqJNFWfDVDkngAA7nOI2A Saved\r\n", 74) = 74 [pid 40676] read(18, "65536\n", 31) = 6 [pid 40676] write(21, "Return-Path: \nReceived: from imap.mydomain.ro\n\tby imap.mydomain.ro (Dovecot) with LMTP id /RqJNFWfDVDkngAA7nOI2A\n\t; Mon, 23 Jul 2012 22:00:37 +0300\nReceived: by imap.mydomain.ro (Postfix, from userid 0)\n\tid AE18F27C687; Mon, 23 Jul 2012 22:00:37 +0300 (EEST)\nDate: Mon, 23 Jul 2012 22:00:37 +0300\nTo: firstuser at anothermydomain.ro, firstuser at mydomain.ro\nCc: anotheruser at mydomain.ro\nSubject: test\nUser-Agent: Heirloom mailx 12.5 6/20/10\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\nMessage-Id: <20120723190037.AE18F27C687 at imap.mydomain.ro>\nFrom: root at imap.mydomain.ro (root)\n\n=\n", 713) = 713 [pid 40676] write(21, "40676:imap", 10) = 10 [pid 40676] write(20, "29 :1343070037.M903400P40676.imap,S=713,W=731\n", 46) = 46 [pid 40676] write(2, "\1\01040676 prefix=lmtp(40676, firstuser): \n", 40) = 40 [pid 40676] write(2, "\1\00240676 /RqJNFWfDVDkngAA7nOI2A: msgid=<20120723190037.AE18F27C687 at imap.mydomain.ro>: saved mail to INBOX\n", 113) = 113 [pid 40676] write(12, "250 2.0.0 /RqJNFWfDVDkngAA7nOI2A Saved\r\n", 72) = 72 [pid 40676] write(21, "Return-Path: \nReceived: from imap.mydomain.ro\n\tby imap.mydomain.ro (Dovecot) with LMTP id /RqJNFWfDVDkngAA7nOI2A\n\t; Mon, 23 Jul 2012 22:00:37 +0300\nReceived: by imap.mydomain.ro (Postfix, from userid 0)\n\tid AE18F27C687; Mon, 23 Jul 2012 22:00:37 +0300 (EEST)\nDate: Mon, 23 Jul 2012 22:00:37 +0300\nTo: firstuser at anothermydomain.ro, firstuser at mydomain.ro\nCc: anotheruser at mydomain.ro\nSubject: test\nUser-Agent: Heirloom mailx 12.5 6/20/10\nMIME-Version: 1.0\nContent-Type: text/plain; charset=us-ascii\nContent-Transfer-Encoding: quoted-printable\nMessage-Id: <20120723190037.AE18F27C687 at imap.mydomain.ro>\nFrom: root at imap.mydomain.ro (root)\n\n=\n", 713) = 713 [pid 40676] write(21, "40676:imap", 10) = 10 [pid 40676] write(20, "30 :fe1a8934559f0d50e49e0000ee7388d8\n", 37) = 37 [pid 40676] write(2, "\1\01040676 prefix=lmtp(40676, firstuser): \n", 40) = 40 [pid 40676] write(2, "\1\00240676 /RqJNFWfDVDkngAA7nOI2A: msgid=<20120723190037.AE18F27C687 at imap.mydomain.ro>: saved mail to INBOX\n", 113) = 113 [pid 40676] write(12, "250 2.0.0 /RqJNFWfDVDkngAA7nOI2A Saved\r\n", 61) = 61 [pid 40676] write(2, "\1\01040676 prefix=lmtp(40676): \n", 29) = 29 [pid 40676] write(2, "\1\00240676 Disconnect from local: Client quit (in reset)\n", 54) = 54 [pid 40676] write(12, "221 2.0.0 Client quit\r\n", 23) = 23 for the sake of readability i stripped elf binary reads and certificates from the strace output. note the dovecot lmtp process reading a single email with three RCPT TO's and writing three times three different files instead of writing content one time and hardlinking two times. anyone care to shed a light? thanks. From lists at wildgooses.com Tue Jul 24 01:45:37 2012 From: lists at wildgooses.com (Ed W) Date: Mon, 23 Jul 2012 23:45:37 +0100 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120719140738.GI24108@charite.de> References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> Message-ID: <500DD411.7050107@wildgooses.com> On 19/07/2012 15:07, Ralf Hildebrandt wrote: > * Ed W : >> On 19/07/2012 13:45, Ralf Hildebrandt wrote: >>> Hi! >>> >>> Anybody got a doveadm script which can remove leading and trailing >>> spaces from folder names? >>> >>> Right now we're migrating mailboxes from dovecot -> Exchange, and >>> Exchange cannot handle leading and trailing spaces in Folder names. >>> >> Caveat that it might cause some wierd symptoms with clients, why not >> attack the dovecot mail backend and rename folders + sed the >> subscription files? > That's what I asked for: > "Anybody got a doveadm script which can remove leading and trailing spaces from folder names?" > Oh, sorry. Why doveadm though? Why not attack the filesystem directly? Ed From fy at 5dshu.com Tue Jul 24 06:58:29 2012 From: fy at 5dshu.com (fy) Date: Tue, 24 Jul 2012 11:58:29 +0800 Subject: [Dovecot] what best for anti-spam filter? Message-ID: <500E1D65.7000808@5dshu.com> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is best ? From gedalya at gedalya.net Tue Jul 24 07:46:16 2012 From: gedalya at gedalya.net (Gedalya) Date: Tue, 24 Jul 2012 00:46:16 -0400 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: <1343105176.4300.1.camel@dml.gedalya.net> It's not what software you use, but what circumstances you run it in. On Tue, 2012-07-24 at 11:58 +0800, fy wrote: > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? From noel.butler at ausics.net Tue Jul 24 07:49:13 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 24 Jul 2012 14:49:13 +1000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: <1343105353.8896.7.camel@tardis> On Tue, 2012-07-24 at 11:58 +0800, fy wrote: > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? amavisd-new with spamassassin and anti virus scanner, clamav with sanesecurity rules use enforcing rules in mail server, like block hosts with no DNS/rDNS Enforce SPF, publish SPF with hardfail use DNSBL's in your mail server, spamhaus, spamcop, spam.sorbs, and more etc. milter regex to stop dynamic/suspect hosts There is no one solution, the solution, is a box of many tricks You might get the odd false blocking, but if system opers can not be bothered running a compliant network with standardised naming conventions for servers, then it is not my problem, and we have had very very very few complaints about this type of policy in over a decade. If someoine sooks to you, educate them, dont whitelist them. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From manu at netbsd.org Tue Jul 24 08:19:00 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Tue, 24 Jul 2012 07:19:00 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> Message-ID: <1knq6r5.wwm2ygzq0sjhM%manu@netbsd.org> fy wrote: > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? milter-greylist of course :-) http://hcpnet.free.fr/milter-greylist/ Note that the name is a tribute to what it has been in the beginning, but we now have much more features than greylisting. IMO the really nice thing in milter-greylist is its ACL, which enable different filtering for different recipients. If you add a user-accessible report about what was filtered, then the users can enable/disable filters on the fly depending on their use, which brings to MTA filtering the interraction we used to have in MUA-based filtering. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu at netbsd.org From p at state-of-mind.de Tue Jul 24 09:16:27 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 24 Jul 2012 08:16:27 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: <20120724061627.GA9237@state-of-mind.de> * fy : > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? The best goes like this: 1. Decide if the SMTP client should be allowed to connect to the server 2. Decide if the client should be allowed to send the message 3. Decide if the message should be allowed to reach the recipient For 1 use e.g. 'postscreen' in Postfix. For 2 use SMTP session filters e.g. smtpd_..._restrictions in Postfix For 3 use a combination of content filters like SpamAssassin, ClamAV etc. In case you need to build some content policies e.g. "recipient A may receive message, messages should never be spam filtered for B and C ..." around the filters use amavisd-new, the content filter framework. It also brings features to manage filtered content e.g. quarantine, copy etc. p at rick -- state of mind () http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From fbianchi at arte.unipi.it Tue Jul 24 09:36:57 2012 From: fbianchi at arte.unipi.it (Federico Bianchi) Date: Tue, 24 Jul 2012 08:36:57 +0200 (CEST) Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: On Tue, 24 Jul 2012, fy wrote: > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? If you can afford using a separate "boundary" SMTP (and, thanks to virtual machines, this is much more common than just a few years ago), MailAvenger is likely to be a very good solution: for Bayesian filtering it relies on plain SpamAssassin, but it really shines in doing deep SMTP transaction analysis, ruling out most spam at that level and making it far less CPU and memory intensive than its counterparts. You can find it at http://mailavenger.org/ Best regards Federico Bianchi Dipartimento di Storia delle Arti Universita` di Pisa via Trieste, 38 - I-56126 Pisa (Italy) tel.(+39) 050 221 6 024; fax (+39) 050 221 6 001 e-mail: =================================================== !DISCLAIMER!: my e-mail reflects _my_own_ opinions! =================================================== From arnaud.abelard at univ-nantes.fr Tue Jul 24 10:16:35 2012 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-15?Q?Arnaud_Ab=E9lard?=) Date: Tue, 24 Jul 2012 09:16:35 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <1343105353.8896.7.camel@tardis> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> Message-ID: <500E4BD3.2060906@univ-nantes.fr> On 07/24/2012 06:49 AM, Noel Butler wrote: > On Tue, 2012-07-24 at 11:58 +0800, fy wrote: > >> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is >> best ? > > > amavisd-new with spamassassin and anti virus scanner, clamav with > sanesecurity rules > use enforcing rules in mail server, like block hosts with no DNS/rDNS > Enforce SPF, publish SPF with hardfail > use DNSBL's in your mail server, spamhaus, spamcop, spam.sorbs, and > more etc. > milter regex to stop dynamic/suspect hosts And first of all, even if this is not dovecot related, use a greylisting solution. > > There is no one solution, the solution, is a box of many tricks > You might get the odd false blocking, but if system opers can not be > bothered running a compliant network with standardised naming > conventions for servers, then it is not my problem, and we have had very > very very few complaints about this type of policy in over a decade. If > someoine sooks to you, educate them, dont whitelist them. Indeed! Fighting spam is a continuous task. While greylisting will cut down the amount of spam by more than 50%, the remaining 50% will give you the hardest time and will keep changing to bypass your rules. You'll need to keep an eye on the flow of false negatives or false positive you are getting... We (72,000 mailboxes) are currently using amavisd-new with spamassassin and CRM114 via a custom plugin instead of the default bayesian filter. Also like Noel, we're using DNSBLs, SPF (although we had to publish a permissive record since some of our users are using their ISP smtp instead of our own). Arnaud > > -- Arnaud Ab?lard (jabber: arnaud.abelard at univ-nantes.fr) Administrateur Syst?me - Responsable Services Web Direction des Syst?mes d'Informations Universit? de Nantes - ne pas utiliser: trapemail at univ-nantes.fr From mailinglists35 at gmail.com Tue Jul 24 10:27:15 2012 From: mailinglists35 at gmail.com (mailing list subscriber) Date: Tue, 24 Jul 2012 10:27:15 +0300 Subject: [Dovecot] Fwd: official dev team position regarding multiple times requested feature (global sieve) In-Reply-To: References: <1343076472.3449.17.camel@linux-nysu.site> Message-ID: forwarding to the proper list address since your reply came with a Reply-To header ---------- Forwarded message ---------- From: mailing list subscriber Date: Tue, Jul 24, 2012 at 10:24 AM Subject: Re: official dev team position regarding multiple times requested feature (global sieve) To: awilliam at opengroupware.us On Mon, Jul 23, 2012 at 11:47 PM, Adam Tauno Williams wrote: > On Mon, 2012-07-23 at 23:26 +0300, mailing list subscriber wrote: >> With above figure in mind, I'm looking at the history of request for >> ability to have a forced, default, site-wide sieve script that the >> user or any other sieve client is unable to alter (I'll just pick most >> relevant): > > Indeed, this would be terribly useful. > >> 2004, with reference to 2001: >> http://goo.gl/Gbo0k >> 2006: A submitted patch >> http://www.irbs.net/internet/info-cyrus/0612/0231.html >> 2007: A very good synthesis: >> http://goo.gl/Lo33b >> 2010: sieve include implemented in v.2.3.0, still fails to meet above >> requirements > > But you fail to reference an open bug?! > The page at http://cyrusimap.web.cmu.edu/mediawiki/index.php/Report_A_Bug says: "If you are absolutely positive you have discovered a bug[...]". This might be misleading since this is a feature request and not a bug. Leaving the technical details of the proper way to report this I'm still curios if the developers are actually AWARE of these repeated requests. >> With all due respect, what is the development's team position >> regarding this feature and how do the development team see a solution >> that meets both requirements? > > I find "SIEVE scripts assigned to folders are not executed" > > > "People forget to cancel their vacation message" > > NOTE: this is really a UI issue, IMO, multiple web clients solve this by > generating intelligent scripts. That's another problem with the poor IT guy trying to assemble bricks from different vendors: when he finally is heard, the answer is "uhm, the feature you requested fail beyond the scope of my brick. I'm sorry, I can't do that", even when there is a proposed patch available! If you read carefully all my referenced links you'll find one that is listing other imap servers that support it. From what I've tested, at least one implementation has done this as a global pre-user execution of a script (dovecot) > > I don't see any bugs about a default-user-sieve-script. *BUT* > imapd.conf does offer this option [IMPLEMENTED]: > autocreate_sieve_script: > The full path of a file that contains a sieve script. This script > automatically becomes a user?s initial default sieve filter script. > When this option is not defined, no default sieve filter is created. > The file must be readable by the cyrus daemon. > > Of course, the user can override this. Unfortunately that feature only addresses PROVISIONING of data, which is quite unuseful since cyrus mail admins do this with the auto-* uoa.gr patches (that's another pain that devs refused to include for a long time) - What it lacks is the ability to handle CHANGE (change of default script, or alter existing user scripts). From stephan at rename-it.nl Tue Jul 24 10:56:23 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Tue, 24 Jul 2012 09:56:23 +0200 Subject: [Dovecot] Fwd: official dev team position regarding multiple times requested feature (global sieve) In-Reply-To: References: <1343076472.3449.17.camel@linux-nysu.site> Message-ID: <500E5527.4040001@rename-it.nl> What is the purpose of this posting? On 7/24/2012 9:27 AM, mailing list subscriber wrote: > forwarding to the proper list address since your reply came with a > Reply-To header Regards, Stephan. From mailinglists35 at gmail.com Tue Jul 24 11:18:14 2012 From: mailinglists35 at gmail.com (mailing list subscriber) Date: Tue, 24 Jul 2012 11:18:14 +0300 Subject: [Dovecot] Fwd: official dev team position regarding multiple times requested feature (global sieve) In-Reply-To: <500E5527.4040001@rename-it.nl> References: <1343076472.3449.17.camel@linux-nysu.site> <500E5527.4040001@rename-it.nl> Message-ID: sorry folks, please ignore me. my head is is spinning trying to get hardlinks and default sieve script working at the same time, writing to dovecot and cyrus at the same time. one is doing the hardlink part good, and the other the sieve. both fail to get both features right at the same time. the previous message was intended for the other imap server :) On Tue, Jul 24, 2012 at 10:56 AM, Stephan Bosch wrote: > What is the purpose of this posting? > > > On 7/24/2012 9:27 AM, mailing list subscriber wrote: >> >> forwarding to the proper list address since your reply came with a >> Reply-To header > > > Regards, > > Stephan. From fy at 5dshu.com Tue Jul 24 11:22:26 2012 From: fy at 5dshu.com (fy) Date: Tue, 24 Jul 2012 16:22:26 +0800 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E4BD3.2060906@univ-nantes.fr> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> Message-ID: <500E5B42.5020609@5dshu.com> ? 2012/7/24 15:16, Arnaud Ab?lard ??: > On 07/24/2012 06:49 AM, Noel Butler wrote: >> On Tue, 2012-07-24 at 11:58 +0800, fy wrote: >> >>> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is >>> best ? >> >> >> amavisd-new with spamassassin and anti virus scanner, clamav with >> sanesecurity rules >> use enforcing rules in mail server, like block hosts with no DNS/rDNS >> Enforce SPF, publish SPF with hardfail >> use DNSBL's in your mail server, spamhaus, spamcop, spam.sorbs, and >> more etc. >> milter regex to stop dynamic/suspect hosts > > And first of all, even if this is not dovecot related, use a > greylisting solution. > >> >> There is no one solution, the solution, is a box of many tricks >> You might get the odd false blocking, but if system opers can not be >> bothered running a compliant network with standardised naming >> conventions for servers, then it is not my problem, and we have had very >> very very few complaints about this type of policy in over a decade. If >> someoine sooks to you, educate them, dont whitelist them. > > Indeed! Fighting spam is a continuous task. While greylisting will cut > down the amount of spam by more than 50%, the remaining 50% will give > you the hardest time and will keep changing to bypass your rules. > You'll need to keep an eye on the flow of false negatives or false > positive you are getting... > > We (72,000 mailboxes) are currently using amavisd-new with > spamassassin and CRM114 via a custom plugin instead of the default > bayesian filter. Also like Noel, we're using DNSBLs, SPF (although we > had to publish a permissive record since some of our users are using > their ISP smtp instead of our own). > > Arnaud > > >> >> > > try slockd ,it is a policy server for postfix http://www.extmail.org/download From robert at schetterer.org Tue Jul 24 11:29:12 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Jul 2012 10:29:12 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E5B42.5020609@5dshu.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5B42.5020609@5dshu.com> Message-ID: <500E5CD8.3040105@schetterer.org> Am 24.07.2012 10:22, schrieb fy: > ? 2012/7/24 15:16, Arnaud Ab?lard ??: >> On 07/24/2012 06:49 AM, Noel Butler wrote: >>> On Tue, 2012-07-24 at 11:58 +0800, fy wrote: >>> >>>> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is >>>> best ? >>> >>> >>> amavisd-new with spamassassin and anti virus scanner, clamav with >>> sanesecurity rules >>> use enforcing rules in mail server, like block hosts with no DNS/rDNS >>> Enforce SPF, publish SPF with hardfail >>> use DNSBL's in your mail server, spamhaus, spamcop, spam.sorbs, and >>> more etc. >>> milter regex to stop dynamic/suspect hosts >> >> And first of all, even if this is not dovecot related, use a >> greylisting solution. >> >>> >>> There is no one solution, the solution, is a box of many tricks >>> You might get the odd false blocking, but if system opers can not be >>> bothered running a compliant network with standardised naming >>> conventions for servers, then it is not my problem, and we have had very >>> very very few complaints about this type of policy in over a decade. If >>> someoine sooks to you, educate them, dont whitelist them. >> >> Indeed! Fighting spam is a continuous task. While greylisting will cut >> down the amount of spam by more than 50%, the remaining 50% will give >> you the hardest time and will keep changing to bypass your rules. >> You'll need to keep an eye on the flow of false negatives or false >> positive you are getting... >> >> We (72,000 mailboxes) are currently using amavisd-new with >> spamassassin and CRM114 via a custom plugin instead of the default >> bayesian filter. Also like Noel, we're using DNSBLs, SPF (although we >> had to publish a permissive record since some of our users are using >> their ISP smtp instead of our own). >> >> Arnaud >> >> >>> >>> >> >> > try slockd ,it is a policy server for postfix > http://www.extmail.org/download sadly there is only poor english info -- Best Regards MfG Robert Schetterer From stan at hardwarefreak.com Tue Jul 24 11:38:00 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 24 Jul 2012 03:38:00 -0500 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E4BD3.2060906@univ-nantes.fr> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> Message-ID: <500E5EE8.4020604@hardwarefreak.com> On 7/24/2012 2:16 AM, Arnaud Ab?lard wrote: > And first of all, even if this is not dovecot related, use a greylisting > solution. Greylisting only stops bots. It is resource intensive, and causes delivery delays. There exist bot spam killing solutions that are just as effective, with less downside. Two are Postfix' postscreen daemon, and fqrdns.pcre, which rejects based on consumer/dynamic looking rDNS. Some users have modified the latter for use on HELO strings instead of client rDNS strings, with good success. Either combined with CBL/ZEN should kill all your bot spam much more efficiently. I'm surprised you're using greylisting (Postgrey?) with 72k mailboxes. > Indeed! Fighting spam is a continuous task. Unfortunately... > We (72,000 mailboxes) are currently using amavisd-new with spamassassin > and CRM114 via a custom plugin instead of the default bayesian filter. > Also like Noel, we're using DNSBLs, SPF (although we had to publish a > permissive record since some of our users are using their ISP smtp > instead of our own). Which of your countermeasures blocks spam from Orange/France Telecom VPS/colo sources? -- Stan From fy at 5dshu.com Tue Jul 24 11:41:58 2012 From: fy at 5dshu.com (fy) Date: Tue, 24 Jul 2012 16:41:58 +0800 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E5CD8.3040105@schetterer.org> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5B42.5020609@5dshu.com> <500E5CD8.3040105@schetterer.org> Message-ID: <500E5FD6.8020808@5dshu.com> ? 2012/7/24 16:29, Robert Schetterer ??: > Am 24.07.2012 10:22, schrieb fy: >> ? 2012/7/24 15:16, Arnaud Ab?lard ??: >>> On 07/24/2012 06:49 AM, Noel Butler wrote: >>>> On Tue, 2012-07-24 at 11:58 +0800, fy wrote: >>>> >>>>> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is >>>>> best ? >>>> >>>> amavisd-new with spamassassin and anti virus scanner, clamav with >>>> sanesecurity rules >>>> use enforcing rules in mail server, like block hosts with no DNS/rDNS >>>> Enforce SPF, publish SPF with hardfail >>>> use DNSBL's in your mail server, spamhaus, spamcop, spam.sorbs, and >>>> more etc. >>>> milter regex to stop dynamic/suspect hosts >>> And first of all, even if this is not dovecot related, use a >>> greylisting solution. >>> >>>> There is no one solution, the solution, is a box of many tricks >>>> You might get the odd false blocking, but if system opers can not be >>>> bothered running a compliant network with standardised naming >>>> conventions for servers, then it is not my problem, and we have had very >>>> very very few complaints about this type of policy in over a decade. If >>>> someoine sooks to you, educate them, dont whitelist them. >>> Indeed! Fighting spam is a continuous task. While greylisting will cut >>> down the amount of spam by more than 50%, the remaining 50% will give >>> you the hardest time and will keep changing to bypass your rules. >>> You'll need to keep an eye on the flow of false negatives or false >>> positive you are getting... >>> >>> We (72,000 mailboxes) are currently using amavisd-new with >>> spamassassin and CRM114 via a custom plugin instead of the default >>> bayesian filter. Also like Noel, we're using DNSBLs, SPF (although we >>> had to publish a permissive record since some of our users are using >>> their ISP smtp instead of our own). >>> >>> Arnaud >>> >>> >>>> >>> >> try slockd ,it is a policy server for postfix >> http://www.extmail.org/download > sadly there is only poor english info > > Download the file for slockd and unzie it . see INSTALL and README files .it's easy use and install . From hsn at filez.com Tue Jul 24 09:20:08 2012 From: hsn at filez.com (Radim Kolar) Date: Tue, 24 Jul 2012 08:20:08 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: <500E3E98.1080608@filez.com> > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? i got best results with dspam + graylist. but dspam is not scalable solution, it works only if you do not have many users. From mstevens at imt-systems.com Tue Jul 24 11:49:48 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 24 Jul 2012 10:49:48 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: <500E4BD3.2060906@univ-nantes.fr> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> Message-ID: <7d028932e10b8e74908866444d61ea3b@imt-systems.com> On 24.07.2012 09:16, Arnaud Ab?lard wrote: > And first of all, even if this is not dovecot related, use a > greylisting solution. No, greylisting is really a bad solution. It is not RFC compliant and delays the mail traffic. I would prefer a pre-queue content-filtering solution like MIMEDefang or amavisd-new. Best regards, Morten From steeeeeveee at gmx.net Tue Jul 24 12:09:52 2012 From: steeeeeveee at gmx.net (Steve) Date: Tue, 24 Jul 2012 11:09:52 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <7d028932e10b8e74908866444d61ea3b@imt-systems.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> Message-ID: <20120724090952.130230@gmx.net> -------- Original-Nachricht -------- > Datum: Tue, 24 Jul 2012 10:49:48 +0200 > Von: Morten Stevens > An: Dovecot Mailing List > Betreff: Re: [Dovecot] what best for anti-spam filter? > On 24.07.2012 09:16, Arnaud Ab?lard wrote: > > > And first of all, even if this is not dovecot related, use a > > greylisting solution. > > No, greylisting is really a bad solution. It is not RFC compliant and > delays the mail traffic. > In what sense is greylisting not RFC compliant? > I would prefer a pre-queue content-filtering solution like MIMEDefang > or amavisd-new. > > Best regards, > > Morten From steeeeeveee at gmx.net Tue Jul 24 12:11:39 2012 From: steeeeeveee at gmx.net (Steve) Date: Tue, 24 Jul 2012 11:11:39 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E3E98.1080608@filez.com> References: <500E1D65.7000808@5dshu.com> <500E3E98.1080608@filez.com> Message-ID: <20120724091139.130260@gmx.net> -------- Original-Nachricht -------- > Datum: Tue, 24 Jul 2012 08:20:08 +0200 > Von: Radim Kolar > An: fy > CC: dovecot at dovecot.org > Betreff: Re: [Dovecot] what best for anti-spam filter? > > > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > > best ? > i got best results with dspam + graylist. but dspam is not scalable > solution, it works only if you do not have many users. > The RICE university is using dspam on about 65K mailboxes without issues: http://it.rice.edu/spam.aspx Maybe your setup is the problem and not dspam? From steeeeeveee at gmx.net Tue Jul 24 12:11:39 2012 From: steeeeeveee at gmx.net (Steve) Date: Tue, 24 Jul 2012 11:11:39 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E3E98.1080608@filez.com> References: <500E1D65.7000808@5dshu.com> <500E3E98.1080608@filez.com> Message-ID: <20120724091139.130260@gmx.net> -------- Original-Nachricht -------- > Datum: Tue, 24 Jul 2012 08:20:08 +0200 > Von: Radim Kolar > An: fy > CC: dovecot at dovecot.org > Betreff: Re: [Dovecot] what best for anti-spam filter? > > > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > > best ? > i got best results with dspam + graylist. but dspam is not scalable > solution, it works only if you do not have many users. > The RICE university is using dspam on about 65K mailboxes without issues: http://it.rice.edu/spam.aspx Maybe your setup is the problem and not dspam? From warren at decoy.co.za Tue Jul 24 12:12:20 2012 From: warren at decoy.co.za (Warren Baker) Date: Tue, 24 Jul 2012 11:12:20 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <7d028932e10b8e74908866444d61ea3b@imt-systems.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> Message-ID: On Tue, Jul 24, 2012 at 10:49 AM, Morten Stevens wrote: > No, greylisting is really a bad solution. It is not RFC compliant and delays > the mail traffic. Since when? RFC5321 was updated to handle delays and then there is RFC6647. -- .warren From dovecot at list-post.mks-mail.de Tue Jul 24 12:36:46 2012 From: dovecot at list-post.mks-mail.de (=?UTF-8?B?TWFya3VzIFNjaMO2bmhhYmVy?=) Date: Tue, 24 Jul 2012 11:36:46 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <7d028932e10b8e74908866444d61ea3b@imt-systems.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> Message-ID: <500E6CAE.7000004@list-post.mks-mail.de> 24.07.2012 10:49, Morten Stevens: > No, greylisting is really a bad solution. It is not RFC compliant and > delays the mail traffic. No. Rejecting mail with an temporary error is perfectly RFC compliant. It happens all the time without greylisting. Because the authors of the SMTP RFC knew that there are many different possible reasons why a mail server might temporarily be unable to receive mail, they added the 4yz responses to it. With greylisting done right, only a very small part of (wanted) incoming mail is delayed. And for the rest: e-mail isn't guaranteed to be a real-time communication channel. -- Regards mks From manu at netbsd.org Tue Jul 24 12:45:50 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Tue, 24 Jul 2012 09:45:50 +0000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E5EE8.4020604@hardwarefreak.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5EE8.4020604@hardwarefreak.com> Message-ID: <20120724094550.GC29783@homeworld.netbsd.org> On Tue, Jul 24, 2012 at 03:38:00AM -0500, Stan Hoeppner wrote: > Greylisting only stops bots. It is resource intensive, and causes > delivery delays. There exist bot spam killing solutions that are just > as effective, with less downside. Two are Postfix' postscreen daemon, > and fqrdns.pcre, which rejects based on consumer/dynamic looking rDNS. I use that in order to decide the greylisting delay: suspect IP get a 12 hours greylist, everyone else gets 15 mn, or 0 if whitelisted by recipeients. It works quite well. -- Emmanuel Dreyfus manu at netbsd.org From manu at netbsd.org Tue Jul 24 12:50:11 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Tue, 24 Jul 2012 09:50:11 +0000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <7d028932e10b8e74908866444d61ea3b@imt-systems.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> Message-ID: <20120724095011.GA5204@homeworld.netbsd.org> On Tue, Jul 24, 2012 at 10:49:48AM +0200, Morten Stevens wrote: > No, greylisting is really a bad solution. It is not RFC compliant Of course it is. Have you readen RFC 6647? > and delays the mail traffic. Greylisting with whitelist and reputation-based greylisting delay makes it painless. -- Emmanuel Dreyfus manu at netbsd.org From hsn at filez.com Tue Jul 24 12:57:18 2012 From: hsn at filez.com (Radim Kolar) Date: Tue, 24 Jul 2012 11:57:18 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <20120724091139.130260@gmx.net> References: <500E1D65.7000808@5dshu.com> <500E3E98.1080608@filez.com> <20120724091139.130260@gmx.net> Message-ID: <500E717E.9050105@filez.com> > The RICE university is using dspam on about 65K mailboxes without issues: http://it.rice.edu/spam.aspx it works with such large number of user only if you use group shared spam/ham dictionaries. From arnaud.abelard at univ-nantes.fr Tue Jul 24 12:57:57 2012 From: arnaud.abelard at univ-nantes.fr (=?UTF-8?B?QXJuYXVkIEFiw6lsYXJk?=) Date: Tue, 24 Jul 2012 11:57:57 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <7d028932e10b8e74908866444d61ea3b@imt-systems.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> Message-ID: <500E71A5.5000406@univ-nantes.fr> On 07/24/2012 10:49 AM, Morten Stevens wrote: > On 24.07.2012 09:16, Arnaud Ab?lard wrote: > >> And first of all, even if this is not dovecot related, use a >> greylisting solution. > > No, greylisting is really a bad solution. It is not RFC compliant and > delays the mail traffic. > > I would prefer a pre-queue content-filtering solution like MIMEDefang or > amavisd-new. > > Best regards, > > Morten As far as I know greylisting is RFC compliant it even has its own RFC: http://tools.ietf.org/html/rfc6647 Greylisting was prefered for a simple reason If a mail is accepted on our servers we have to deliver it to the user (unless it has a virus) - With greylisting we aren't rejecting potentially spammy mails, we are rejecting misbehaving servers. That's important, legally speaking. We could be in trouble if we rejected an important mail by mistake when our server actually accepted it. - Our users already complain they are getting too much spams (we mark them with [SPAM] in the subject). Without greylist and for the reason I stated above, even if we were able to detect the bot spam as spams, we wouldn't want to reject it and our users would be flooded. Greylisting isn't a perfect solution. Just a good one. Depends on what you want, I guess. Arnaud -- Arnaud Ab?lard (jabber: arnaud.abelard at univ-nantes.fr) Administrateur Syst?me - Responsable Services Web Direction des Syst?mes d'Informations Universit? de Nantes - ne pas utiliser: trapemail at univ-nantes.fr From arnaud.abelard at univ-nantes.fr Tue Jul 24 12:58:57 2012 From: arnaud.abelard at univ-nantes.fr (=?ISO-8859-1?Q?Arnaud_Ab=E9lard?=) Date: Tue, 24 Jul 2012 11:58:57 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E5EE8.4020604@hardwarefreak.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5EE8.4020604@hardwarefreak.com> Message-ID: <500E71E1.7010800@univ-nantes.fr> On 07/24/2012 10:38 AM, Stan Hoeppner wrote: > On 7/24/2012 2:16 AM, Arnaud Ab?lard wrote: > >> And first of all, even if this is not dovecot related, use a greylisting >> solution. > > Greylisting only stops bots. It is resource intensive, and causes > delivery delays. There exist bot spam killing solutions that are just > as effective, with less downside. Two are Postfix' postscreen daemon, > and fqrdns.pcre, which rejects based on consumer/dynamic looking rDNS. > Some users have modified the latter for use on HELO strings instead of > client rDNS strings, with good success. Either combined with CBL/ZEN > should kill all your bot spam much more efficiently. I'm surprised > you're using greylisting (Postgrey?) with 72k mailboxes. Greylisting only stops bots. Exactly. That's the whole point! We have been using sqlgrey for now 5 years and we only had one problem last month with OVH smtp infrastructure which sucks and we're happy to see mails bouncing from them, hoping their customers will complain. But I can understand why you would think greylist is trouble. It depends on how you set it up. One mail delayed per domain and per month is really nothing compared to hundred thousands of bot spams we are rejecting. dynamic/consumer ip range DNSBL are dangerous since they are rarely up to date, I can painfully remember that. I guess it all depends on what kind of smtp traffic you get. As a large university we aren't getting the same traffic as a big corporate company which will mostly communicate with other business. We are getting tons of individual mails from local ISPs, lot of geeks hosting their servers at home (a lot of ppl do that here...), etc. >> Indeed! Fighting spam is a continuous task. > > Unfortunately... > >> We (72,000 mailboxes) are currently using amavisd-new with spamassassin >> and CRM114 via a custom plugin instead of the default bayesian filter. >> Also like Noel, we're using DNSBLs, SPF (although we had to publish a >> permissive record since some of our users are using their ISP smtp >> instead of our own). > > Which of your countermeasures blocks spam from Orange/France Telecom > VPS/colo sources? > Ahah.. that's a good question! since we are a french university we are also getting tons of clean mails from Orange/FT. But the problem isn't as bad as it used to be since Orange is now blocking direct outgoing traffic on port 25 for a few years now. Back then the DNSRBL were a good solution for spams coming from them. Now the new pain in the ass is OVH, the largest european hosting company which also has the worst smtp infrastructure that will not play well with greylist (tons of smtp servers, each on a different ip range so you can't even whitelist them by their networks). Arnaud -- Arnaud Ab?lard (jabber: arnaud.abelard at univ-nantes.fr) Administrateur Syst?me - Responsable Services Web Direction des Syst?mes d'Informations Universit? de Nantes - ne pas utiliser: trapemail at univ-nantes.fr From andrzej.filip at gmail.com Tue Jul 24 13:03:55 2012 From: andrzej.filip at gmail.com (Andrzej A. Filip) Date: Tue, 24 Jul 2012 12:03:55 +0200 Subject: [Dovecot] what best for anti-spam filter? [greylisting] In-Reply-To: <20120724094550.GC29783@homeworld.netbsd.org> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5EE8.4020604@hardwarefreak.com> <20120724094550.GC29783@homeworld.netbsd.org> Message-ID: <500E730B.5040003@gmail.com> On 07/24/2012 11:45 AM, Emmanuel Dreyfus wrote: > On Tue, Jul 24, 2012 at 03:38:00AM -0500, Stan Hoeppner wrote: >> Greylisting only stops bots. It is resource intensive, and causes >> delivery delays. There exist bot spam killing solutions that are just >> as effective, with less downside. Two are Postfix' postscreen daemon, >> and fqrdns.pcre, which rejects based on consumer/dynamic looking rDNS. > I use that in order to decide the greylisting delay: suspect IP get a > 12 hours greylist, everyone else gets 15 mn, or 0 if whitelisted by > recipeients. It works quite well. Have you considered using some dnswl (whitelist) to turn off greylisting for some hosts? e.g. dnswl.org. Greylisting for dnswl.org "none" level (the lowest trust) makes (some) sense only if you use bulk detectors like razor/pyzor/DCC. From manu at netbsd.org Tue Jul 24 13:19:50 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Tue, 24 Jul 2012 10:19:50 +0000 Subject: [Dovecot] what best for anti-spam filter? [greylisting] In-Reply-To: <500E730B.5040003@gmail.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5EE8.4020604@hardwarefreak.com> <20120724094550.GC29783@homeworld.netbsd.org> <500E730B.5040003@gmail.com> Message-ID: <20120724101950.GB5204@homeworld.netbsd.org> On Tue, Jul 24, 2012 at 12:03:55PM +0200, Andrzej A. Filip wrote: > Have you considered using some dnswl (whitelist) to turn off greylisting > for some hosts? I do not do it, but it is trivial to configure milter-greylist for that usage: just add a whitelist acl based on a DNSRBL lookup. -- Emmanuel Dreyfus manu at netbsd.org From mstevens at imt-systems.com Tue Jul 24 13:27:30 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 24 Jul 2012 12:27:30 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: <20120724095011.GA5204@homeworld.netbsd.org> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <20120724095011.GA5204@homeworld.netbsd.org> Message-ID: <58d272c40480f0dad57fe2e6d310e4ff@imt-systems.com> On 24.07.2012 11:50, Emmanuel Dreyfus wrote: > On Tue, Jul 24, 2012 at 10:49:48AM +0200, Morten Stevens wrote: >> No, greylisting is really a bad solution. It is not RFC compliant > > Of course it is. Have you readen RFC 6647? Okay, you're right. Here is it from June 2012: http://www.ietf.org/mail-archive/web/apps-discuss/current/msg06288.html So it is now RFC compliant. Anyway I think delaying mail traffic is not a good solution. Best regards, Morten From noel.butler at ausics.net Tue Jul 24 13:33:53 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 24 Jul 2012 20:33:53 +1000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> Message-ID: <1343126033.3802.3.camel@tardis> On Tue, 2012-07-24 at 11:12 +0200, Warren Baker wrote: > On Tue, Jul 24, 2012 at 10:49 AM, Morten Stevens > wrote: > > No, greylisting is really a bad solution. It is not RFC compliant and delays > > the mail traffic. > > Since when? RFC5321 was updated to handle delays and then there is RFC6647. > When we looked at it years ago, it did little to stem the tide of spam, all it did was over give us a negative impact by delays of legit mail, some servers are also poorly configured and dont try resend for a long period of time, especially if their queue is also loaded up, then there's non compliant twits like a certain freemail service that does not retry, period. Too many downfalls, it lasted all of a few days when we tried it out. It might be acceptable for small SOHO, but it is unacceptable for ISP's and hosting companies who process millions of mails a day. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From steeeeeveee at gmx.net Tue Jul 24 13:39:21 2012 From: steeeeeveee at gmx.net (Steve) Date: Tue, 24 Jul 2012 12:39:21 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E717E.9050105@filez.com> References: <500E1D65.7000808@5dshu.com> <500E3E98.1080608@filez.com> <20120724091139.130260@gmx.net> <500E717E.9050105@filez.com> Message-ID: <20120724103921.130250@gmx.net> -------- Original-Nachricht -------- > Datum: Tue, 24 Jul 2012 11:57:18 +0200 > Von: Radim Kolar > An: Dovecot Mailing List > Betreff: Re: [Dovecot] what best for anti-spam filter? > > The RICE university is using dspam on about 65K mailboxes without > issues: http://it.rice.edu/spam.aspx > it works with such large number of user only if you use group shared > spam/ham dictionaries. > I don't remember Kenneth Marshall saying anything about using shared groups when describing his setup. As far as I know they use an older version of dspam and don't use group support at all. Their backend is PostgreSQL. From jtam.home at gmail.com Tue Jul 24 13:51:05 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Tue, 24 Jul 2012 03:51:05 -0700 (PDT) Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: References: Message-ID: Morten Stevens writes: > So it is now RFC compliant. Anyway I think delaying mail traffic is not > a good solution. Well, OK, if you not keen on greylisting, you can try greet pausing, which introduces a shorter delay. It tests a bot's patience by inserting a pre-HELO pause. RFC allows 5 minutes before timeout. This is last week's stat for one of my mail server and the count of early-talker or early-disconnecter, almost all of which are bots. A greet pause of over 20s dumps a lot of bots. Expect to whitelist the odd server here and there because they've tuned their servers to some aggressively small RFC non-compliant timeouts. Delay Disconnect/early-talkers 00 = 9 01 = 1 02 = 1 03 = 11 04 = 34 05 = 5 06 = 1 07 = 1 08 = 3 09 = 3 10 = 9 11 = 1 12 = 2 16 = 7 17 = 10 18 = 4 19 = 17 20 = 1161 21 = 431 22 = 61 23 = 43 24 = 13 Joseph Tam From mstevens at imt-systems.com Tue Jul 24 13:51:53 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 24 Jul 2012 12:51:53 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: <1343126033.3802.3.camel@tardis> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <1343126033.3802.3.camel@tardis> Message-ID: <0ffe0d77dacfc37787b8f258b448ec07@imt-systems.com> On 24.07.2012 12:33, Noel Butler wrote: > When we looked at it years ago, it did little to stem the tide of > spam, > all it did was over give us a negative impact by delays of legit > mail, > some servers are also poorly configured and dont try resend for a > long > period of time, especially if their queue is also loaded up, then > there's non compliant twits like a certain freemail service that does > not retry, period. Too many downfalls, it lasted all of a few days > when > we tried it out. It might be acceptable for small SOHO, but it is > unacceptable for ISP's and hosting companies who process millions of > mails a day. Excellent post, Noel. That's what I mean. Best regards, Morten From warren at decoy.co.za Tue Jul 24 14:00:30 2012 From: warren at decoy.co.za (Warren Baker) Date: Tue, 24 Jul 2012 13:00:30 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: References: Message-ID: On Tue, Jul 24, 2012 at 12:51 PM, Joseph Tam wrote: > > Well, OK, if you not keen on greylisting, you can try greet pausing, > which introduces a shorter delay. > This, works well. Interesting your sweet spot is around 20seconds, I found 13s to be the right mark. -- .warren From mstevens at imt-systems.com Tue Jul 24 14:18:18 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 24 Jul 2012 13:18:18 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: References: Message-ID: <20f088c53dc0e45190712b21e0d5a7e6@imt-systems.com> On 24.07.2012 12:51, Joseph Tam wrote: > Morten Stevens writes: > >> So it is now RFC compliant. Anyway I think delaying mail traffic is >> not >> a good solution. > > Well, OK, if you not keen on greylisting, you can try greet pausing, > which introduces a shorter delay. > > It tests a bot's patience by inserting a pre-HELO pause. RFC allows > 5 > minutes before timeout. This is last week's stat for one of my mail > server and the count of early-talker or early-disconnecter, almost > all of which are bots. A greet pause of over 20s dumps a lot of > bots. > Expect to whitelist the odd server here and there because they've > tuned > their servers to some aggressively small RFC non-compliant timeouts. Yes, something like greet_pause (sendmail) or postscreen (without deep protocol tests) is a very good solution. In addition, several DNSBLs with different scores. This could for example look like this: (for postfix users) postscreen_dnsbl_threshold = 3 postscreen_dnsbl_action = drop postscreen_greet_action = enforce postscreen_dnsbl_sites = ix.dnsbl.manitu.net*3 b.barracudacentral.org*3 zen.spamhaus.org*3 dnsbl.njabl.org*2 bl.spameatingmonkey.net*2 bl.spamcop.net spamtrap.trblspam.com list.dnswl.org=127.[0..255].[0..255].0*-2 list.dnswl.org=127.[0..255].[0..255].1*-4 list.dnswl.org=127.[0..255].[0..255].[2..255]*-6 Best regards, Morten From robert at schetterer.org Tue Jul 24 14:24:30 2012 From: robert at schetterer.org (Robert Schetterer) Date: Tue, 24 Jul 2012 13:24:30 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E5FD6.8020808@5dshu.com> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <500E5B42.5020609@5dshu.com> <500E5CD8.3040105@schetterer.org> <500E5FD6.8020808@5dshu.com> Message-ID: <500E85EE.8040205@schetterer.org> Am 24.07.2012 10:41, schrieb fy: > ? 2012/7/24 16:29, Robert Schetterer ??: >> Am 24.07.2012 10:22, schrieb fy: >>> ? 2012/7/24 15:16, Arnaud Ab?lard ??: >>>> On 07/24/2012 06:49 AM, Noel Butler wrote: >>>>> On Tue, 2012-07-24 at 11:58 +0800, fy wrote: >>>>> >>>>>> what anti-spam for you used ? dspam?spammassian? amavisd-new ? >>>>>> what is >>>>>> best ? >>>>> >>>>> amavisd-new with spamassassin and anti virus scanner, clamav with >>>>> sanesecurity rules >>>>> use enforcing rules in mail server, like block hosts with no DNS/rDNS >>>>> Enforce SPF, publish SPF with hardfail >>>>> use DNSBL's in your mail server, spamhaus, spamcop, spam.sorbs, and >>>>> more etc. >>>>> milter regex to stop dynamic/suspect hosts >>>> And first of all, even if this is not dovecot related, use a >>>> greylisting solution. >>>> >>>>> There is no one solution, the solution, is a box of many tricks >>>>> You might get the odd false blocking, but if system opers can not be >>>>> bothered running a compliant network with standardised naming >>>>> conventions for servers, then it is not my problem, and we have had >>>>> very >>>>> very very few complaints about this type of policy in over a >>>>> decade. If >>>>> someoine sooks to you, educate them, dont whitelist them. >>>> Indeed! Fighting spam is a continuous task. While greylisting will cut >>>> down the amount of spam by more than 50%, the remaining 50% will give >>>> you the hardest time and will keep changing to bypass your rules. >>>> You'll need to keep an eye on the flow of false negatives or false >>>> positive you are getting... >>>> >>>> We (72,000 mailboxes) are currently using amavisd-new with >>>> spamassassin and CRM114 via a custom plugin instead of the default >>>> bayesian filter. Also like Noel, we're using DNSBLs, SPF (although we >>>> had to publish a permissive record since some of our users are using >>>> their ISP smtp instead of our own). >>>> >>>> Arnaud >>>> >>>> >>>>> >>>> >>> try slockd ,it is a policy server for postfix >>> http://www.extmail.org/download >> sadly there is only poor english info >> >> > Download the file for slockd and unzie it . see INSTALL and README files > .it's easy use and install . done , nice tool -- Best Regards MfG Robert Schetterer From manu at netbsd.org Tue Jul 24 14:44:59 2012 From: manu at netbsd.org (Emmanuel Dreyfus) Date: Tue, 24 Jul 2012 13:44:59 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <58d272c40480f0dad57fe2e6d310e4ff@imt-systems.com> Message-ID: <1knqouc.t8bxqe9ivf1M%manu@netbsd.org> Morten Stevens wrote: > So it is now RFC compliant. Anyway I think delaying mail traffic is not > a good solution. This is why whitelists and autowhilists are used in greylist filters. -- Emmanuel Dreyfus http://hcpnet.free.fr/pubz manu at netbsd.org From dovecot at list-post.mks-mail.de Tue Jul 24 15:06:22 2012 From: dovecot at list-post.mks-mail.de (=?UTF-8?B?TWFya3VzIFNjaMO2bmhhYmVy?=) Date: Tue, 24 Jul 2012 14:06:22 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E71A5.5000406@univ-nantes.fr> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> Message-ID: <500E8FBE.5000204@list-post.mks-mail.de> 24.07.2012 11:57, Arnaud Ab?lard: > - With greylisting we aren't rejecting potentially spammy mails, we are > rejecting misbehaving servers. That's important, legally speaking. We > could be in trouble if we rejected an important mail by mistake when our > server actually accepted it. That's something which is not greylisting-specific at all. You must not accept mail you are unwilling or unable to deliver - ever! Creating bounces will make you a source of backscatter and get you blacklisted, eventually. ("Outgoing" mail is a different matter, of course) But that doesn't mean that greylisting is the only means for fighting spam that is compliant to the above rule. It's, for example, not uncommon to have things like milters or pre-queue filters pipe the incoming mail through a spam checker and accept or reject the mail - during the SMTP dialogue - depending on the result of the check. -- Regards mks From CMarcus at Media-Brokers.com Tue Jul 24 15:11:04 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 24 Jul 2012 08:11:04 -0400 Subject: [Dovecot] maildir_copy_with_hardlinks on v.2.0.19 In-Reply-To: References: Message-ID: <500E90D8.7060807@Media-Brokers.com> On 2012-07-23 3:12 PM, mailing list subscriber wrote: > On Sun, Jul 22, 2012 at 2:59 PM, mailing list subscriber wrote: >> I'm trying to get the so-called "single instance store" (I think cyrus >> has got the name for the first time) with dovecot --version = 2.0.19 >> binary package installed from ubuntu 12.04 lts official repo. > As requested, here is my update. As you can see I am running now the > latest release however emails delivered through lmtp gets split into > different files instead of expected hardlinked files. > > Please pay attention as I'm commenting in-between different pasted > outputs: > > # 2.1.8 (30b0d6b1c581): /etc/dovecot/dovecot.conf Well, apparently you haven't enabled SIS - it requires explicitly setting "mail_attachment_dir" (also be aware, this only applies to *attachments*, the messages themselves (email body, headers, etc) are still stored separately), but I discovered that this doesn't seem to be documented in the wiki either, so not surprising you missed it - I only found one reference in one email list thread: http://www.dovecot.org/list/dovecot/2012-March/064390.html Hth... -- Best regards, Charles From mstevens at imt-systems.com Tue Jul 24 15:13:41 2012 From: mstevens at imt-systems.com (Morten Stevens) Date: Tue, 24 Jul 2012 14:13:41 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: <1knqouc.t8bxqe9ivf1M%manu@netbsd.org> References: <1knqouc.t8bxqe9ivf1M%manu@netbsd.org> Message-ID: <1512ca7ae2726559c866cbec3f172dad@imt-systems.com> On 24.07.2012 13:44, manu at netbsd.org wrote: > Morten Stevens wrote: > >> So it is now RFC compliant. Anyway I think delaying mail traffic is >> not >> a good solution. > > This is why whitelists and autowhilists are used in greylist filters. Okay, and where are your whitelists at netbsd.org? I've sent an email to you, which is delayed until now. Note: Our servers are listed at dnswl.org with medium score. Jul 24 12:27:32 mx1 sendmail[31933]: q6OARUOM031928: to=, delay=00:00:02, xdelay=00:00:01, mailer=esmtp, pri=152317, relay=dovecot.org. [193.210.130.67], dsn=2.0.0, stat=Sent (Ok: queued as 35AF81AE8359) Jul 24 12:28:32 mx1 sendmail[31933]: q6OARUOM031928: to=, delay=00:01:02, xdelay=00:01:00, mailer=esmtp, pri=152317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.0.0, stat=Deferred: Connection timed out with mail.netbsd.org. Jul 24 12:42:57 mx1 sendmail[32292]: q6OARUOM031928: to=, delay=00:15:27, xdelay=00:01:00, mailer=esmtp, pri=242317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.0.0, stat=Deferred: Connection timed out with mail.netbsd.org. Jul 24 12:50:53 mx1 sendmail[32518]: q6OARUOM031928: to=, delay=00:23:23, xdelay=00:00:02, mailer=esmtp, pri=332317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, stat=Deferred: 450 4.7.1 : Recipient address rejected: Greylisting in action, please try later Jul 24 12:58:41 mx1 sendmail[312]: q6OARUOM031928: to=, delay=00:31:11, xdelay=00:00:02, mailer=esmtp, pri=422317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, stat=Deferred: 450 4.7.1 : Recipient address rejected: Greylisting in action, please try later Jul 24 13:42:21 mx1 sendmail[1461]: q6OARUOM031928: to=, delay=01:14:51, xdelay=00:00:01, mailer=esmtp, pri=512317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, stat=Deferred: 450 4.7.1 : Recipient address rejected: Greylisting in action, please try later Jul 24 13:50:53 mx1 sendmail[1672]: q6OARUOM031928: to=, delay=01:23:23, xdelay=00:00:02, mailer=esmtp, pri=602317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, stat=Deferred: 450 4.7.1 : Recipient address rejected: Greylisting in action, please try later This is exactly the reason why greylisting is bad. Best regards, Morten From stan at hardwarefreak.com Tue Jul 24 15:22:38 2012 From: stan at hardwarefreak.com (Stan Hoeppner) Date: Tue, 24 Jul 2012 07:22:38 -0500 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <1512ca7ae2726559c866cbec3f172dad@imt-systems.com> References: <1knqouc.t8bxqe9ivf1M%manu@netbsd.org> <1512ca7ae2726559c866cbec3f172dad@imt-systems.com> Message-ID: <500E938E.5090203@hardwarefreak.com> On 7/24/2012 7:13 AM, Morten Stevens wrote: > Jul 24 12:27:32 mx1 sendmail[31933]: q6OARUOM031928: > to=, delay=00:00:02, xdelay=00:00:01, mailer=esmtp, > pri=152317, relay=dovecot.org. [193.210.130.67], dsn=2.0.0, stat=Sent > (Ok: queued as 35AF81AE8359) > Jul 24 12:28:32 mx1 sendmail[31933]: q6OARUOM031928: > to=, delay=00:01:02, xdelay=00:01:00, mailer=esmtp, > pri=152317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.0.0, > stat=Deferred: Connection timed out with mail.netbsd.org. > Jul 24 12:42:57 mx1 sendmail[32292]: q6OARUOM031928: > to=, delay=00:15:27, xdelay=00:01:00, mailer=esmtp, > pri=242317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.0.0, > stat=Deferred: Connection timed out with mail.netbsd.org. > Jul 24 12:50:53 mx1 sendmail[32518]: q6OARUOM031928: > to=, delay=00:23:23, xdelay=00:00:02, mailer=esmtp, > pri=332317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, > stat=Deferred: 450 4.7.1 : Recipient address rejected: > Greylisting in action, please try later > Jul 24 12:58:41 mx1 sendmail[312]: q6OARUOM031928: to=, > delay=00:31:11, xdelay=00:00:02, mailer=esmtp, pri=422317, > relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, stat=Deferred: 450 > 4.7.1 : Recipient address rejected: Greylisting in > action, please try later > Jul 24 13:42:21 mx1 sendmail[1461]: q6OARUOM031928: > to=, delay=01:14:51, xdelay=00:00:01, mailer=esmtp, > pri=512317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, > stat=Deferred: 450 4.7.1 : Recipient address rejected: > Greylisting in action, please try later > Jul 24 13:50:53 mx1 sendmail[1672]: q6OARUOM031928: > to=, delay=01:23:23, xdelay=00:00:02, mailer=esmtp, > pri=602317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, > stat=Deferred: 450 4.7.1 : Recipient address rejected: > Greylisting in action, please try later > > This is exactly the reason why greylisting is bad. I have yet to hear of a bot that retries. Thus, there's not reason to set a wait period more than a few seconds, causing the situation above. That said, there is another use of greylisting not related to bots, which is delaying clients long periods of time in hopes that snowshoe servers will get listed by one's fav dnsbl. Though this isn't very effective against snowshoe. Which is why few use it for this purpose. -- Stan From CMarcus at Media-Brokers.com Tue Jul 24 15:30:45 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Tue, 24 Jul 2012 08:30:45 -0400 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E4BD3.2060906@univ-nantes.fr> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> Message-ID: <500E9575.5000100@Media-Brokers.com> On 2012-07-24 3:16 AM, Arnaud Ab?lard wrote: > And first of all, even if this is not dovecot related, use a greylisting > solution. Correct me if I'm wrong, but doesn't postscreen mostly make greylisting unnecessary? -- Best regards, Charles From noel.butler at ausics.net Tue Jul 24 15:30:26 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 24 Jul 2012 22:30:26 +1000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E8FBE.5000204@list-post.mks-mail.de> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> Message-ID: <1343133026.5634.6.camel@tardis> On Tue, 2012-07-24 at 14:06 +0200, Markus Sch?nhaber wrote: > You must not accept mail you are unwilling or unable to deliver - ever! That insisted behaviour was changed four years ago, read up on RFC 5321 IIRC -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From noel.butler at ausics.net Tue Jul 24 15:36:23 2012 From: noel.butler at ausics.net (Noel Butler) Date: Tue, 24 Jul 2012 22:36:23 +1000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E938E.5090203@hardwarefreak.com> References: <1knqouc.t8bxqe9ivf1M%manu@netbsd.org> <1512ca7ae2726559c866cbec3f172dad@imt-systems.com> <500E938E.5090203@hardwarefreak.com> Message-ID: <1343133383.5634.11.camel@tardis> On Tue, 2012-07-24 at 07:22 -0500, Stan Hoeppner wrote: > On 7/24/2012 7:13 AM, Morten Stevens wrote: > > > Jul 24 12:27:32 mx1 sendmail[31933]: q6OARUOM031928: > > to=, delay=00:00:02, xdelay=00:00:01, mailer=esmtp, > > pri=152317, relay=dovecot.org. [193.210.130.67], dsn=2.0.0, stat=Sent > > (Ok: queued as 35AF81AE8359) > > Jul 24 12:28:32 mx1 sendmail[31933]: q6OARUOM031928: > > to=, delay=00:01:02, xdelay=00:01:00, mailer=esmtp, > > pri=152317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.0.0, > > stat=Deferred: Connection timed out with mail.netbsd.org. > > Jul 24 12:42:57 mx1 sendmail[32292]: q6OARUOM031928: > > to=, delay=00:15:27, xdelay=00:01:00, mailer=esmtp, > > pri=242317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.0.0, > > stat=Deferred: Connection timed out with mail.netbsd.org. > > Jul 24 12:50:53 mx1 sendmail[32518]: q6OARUOM031928: > > to=, delay=00:23:23, xdelay=00:00:02, mailer=esmtp, > > pri=332317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, > > stat=Deferred: 450 4.7.1 : Recipient address rejected: > > Greylisting in action, please try later > > Jul 24 12:58:41 mx1 sendmail[312]: q6OARUOM031928: to=, > > delay=00:31:11, xdelay=00:00:02, mailer=esmtp, pri=422317, > > relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, stat=Deferred: 450 > > 4.7.1 : Recipient address rejected: Greylisting in > > action, please try later > > Jul 24 13:42:21 mx1 sendmail[1461]: q6OARUOM031928: > > to=, delay=01:14:51, xdelay=00:00:01, mailer=esmtp, > > pri=512317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, > > stat=Deferred: 450 4.7.1 : Recipient address rejected: > > Greylisting in action, please try later > > Jul 24 13:50:53 mx1 sendmail[1672]: q6OARUOM031928: > > to=, delay=01:23:23, xdelay=00:00:02, mailer=esmtp, > > pri=602317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, > > stat=Deferred: 450 4.7.1 : Recipient address rejected: > > Greylisting in action, please try later > > > > This is exactly the reason why greylisting is bad. > > I have yet to hear of a bot that retries. Thus, there's not reason to they exist > set a wait period more than a few seconds, causing the situation above. > Not surprising Stanley still doesn't get it, how many mail servers have a retry under 10 minutes anyway, sure as hell none at a "few seconds" , and busy servers with queue limits, could mean a msg set to be retried in 10 mins, may still yet take 2 hours to find its spot in a queue (yes, seen it) Sendmails greet pause as was noted by an earlier poster was not a bad method, but like grey listing, some spam bots adjusted to suit. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From arnaud.abelard at univ-nantes.fr Tue Jul 24 16:11:37 2012 From: arnaud.abelard at univ-nantes.fr (=?UTF-8?B?QXJuYXVkIEFiw6lsYXJk?=) Date: Tue, 24 Jul 2012 15:11:37 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E8FBE.5000204@list-post.mks-mail.de> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> Message-ID: <500E9F09.7030503@univ-nantes.fr> On 07/24/2012 02:06 PM, Markus Sch?nhaber wrote: > 24.07.2012 11:57, Arnaud Ab?lard: > >> - With greylisting we aren't rejecting potentially spammy mails, we are >> rejecting misbehaving servers. That's important, legally speaking. We >> could be in trouble if we rejected an important mail by mistake when our >> server actually accepted it. > > That's something which is not greylisting-specific at all. You must not > accept mail you are unwilling or unable to deliver - ever! That's my point. Greylisting screens bad behaviored servers away and if a mail is accepted it will be delivered. If it's detected as a potential spam, it will still be delivered to the end user with a proper tag in the subject. From what I just read, it seems that indeed postscreen could be an alternative for that purpose. But screening solutions aren't enough since a lot of unwanted mails are sent from legit RFC compliant servers: - newsletters from sites the users provided their email to and forgot they did. - digital prospecting which is legal if properly done (in France, it must be related to your professionnal field of activity and an unsubscribe link must be provided) - phishing and scams sent from stolen webmail accounts. Greylisting and DNSBL aren't really useful for any of those, only content analysis will catch them and that's the hard part. Bayesian and markovian filters need training and corrections. Spamassassin rules needs to be added every few weeks, etc. I kind of like how pyzor and razor work but those are rather slow and tend to use too much CPU. Anyone here who had a good experience with those? Arnaud > Creating bounces will make you a source of backscatter and get you > blacklisted, eventually. > ("Outgoing" mail is a different matter, of course) > > But that doesn't mean that greylisting is the only means for fighting > spam that is compliant to the above rule. It's, for example, not > uncommon to have things like milters or pre-queue filters pipe the > incoming mail through a spam checker and accept or reject the mail - > during the SMTP dialogue - depending on the result of the check. > -- Arnaud Ab?lard (jabber: arnaud.abelard at univ-nantes.fr) Administrateur Syst?me - Responsable Services Web Direction des Syst?mes d'Informations Universit? de Nantes - ne pas utiliser: trapemail at univ-nantes.fr From dovecot at list-post.mks-mail.de Tue Jul 24 16:31:28 2012 From: dovecot at list-post.mks-mail.de (=?ISO-8859-15?Q?Markus_Sch=F6nhaber?=) Date: Tue, 24 Jul 2012 15:31:28 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <1343133026.5634.6.camel@tardis> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> <1343133026.5634.6.camel@tardis> Message-ID: <500EA3B0.40705@list-post.mks-mail.de> 24.07.2012 14:30, Noel Butler: > On Tue, 2012-07-24 at 14:06 +0200, Markus Sch?nhaber wrote: > >> You must not accept mail you are unwilling or unable to deliver - ever! > > > That insisted behaviour was changed four years ago, read up on RFC 5321 Where does it say so? > IIRC I doubt you do. -- Regards mks From p at state-of-mind.de Tue Jul 24 16:46:27 2012 From: p at state-of-mind.de (Patrick Ben Koetter) Date: Tue, 24 Jul 2012 15:46:27 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E9F09.7030503@univ-nantes.fr> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> <500E9F09.7030503@univ-nantes.fr> Message-ID: <20120724134627.GF18334@state-of-mind.de> People, this is a mailing list dedicated to Dovecot and the protocols POP, IMAP and MANAGESIEVE with the one or the other detour to storage. Greylisting and other Anti-Spam techniques, as discussed in this thread, truely are off-topic. Please take discussion offlist or to another list that deals with such stuff. p at rick -- state of mind () Digitale Kommunikation http://www.state-of-mind.de Franziskanerstra?e 15 Telefon +49 89 3090 4664 81669 M?nchen Telefax +49 89 3090 4666 Amtsgericht M?nchen Partnerschaftsregister PR 563 From joshua at hybrid.pl Tue Jul 24 17:23:28 2012 From: joshua at hybrid.pl (Jacek Osiecki) Date: Tue, 24 Jul 2012 16:23:28 +0200 (CEST) Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E938E.5090203@hardwarefreak.com> References: <1knqouc.t8bxqe9ivf1M%manu@netbsd.org> <1512ca7ae2726559c866cbec3f172dad@imt-systems.com> <500E938E.5090203@hardwarefreak.com> Message-ID: On Tue, 24 Jul 2012, Stan Hoeppner wrote: > On 7/24/2012 7:13 AM, Morten Stevens wrote: [...] >> Jul 24 12:50:53 mx1 sendmail[32518]: q6OARUOM031928: >> to=, delay=00:23:23, xdelay=00:00:02, mailer=esmtp, >> pri=332317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, >> stat=Deferred: 450 4.7.1 : Recipient address rejected: >> Greylisting in action, please try later [...] >> Greylisting in action, please try later >> Jul 24 13:50:53 mx1 sendmail[1672]: q6OARUOM031928: >> to=, delay=01:23:23, xdelay=00:00:02, mailer=esmtp, >> pri=602317, relay=mail.netbsd.org. [149.20.53.66], dsn=4.7.1, >> stat=Deferred: 450 4.7.1 : Recipient address rejected: >> Greylisting in action, please try later >> This is exactly the reason why greylisting is bad. I'd say, when greylisting isn't set up correctly. One hour and still greylisting the message? Come on... > I have yet to hear of a bot that retries. Thus, there's not reason to > set a wait period more than a few seconds, causing the situation above. Few seconds is much too short. One of our clients has over 20 servers accross the country, with central GL database. Most of them are MX for the domain, and each one is a storage for some subset of emails in this domain. When a spambot tries to deliver a message, goes through all the MXes - so it takes sometimes 20-30seconds for it to get through all of them... The initial pre-greeting delay is a good idea - although IMHO users definitely should then use submission port (587) without this delay. For GL, there is no point in setting times larger than few minutes. Bots either don't retry to send email at all, or retry in legit times. On the other hands, most of the spoiled mail servers (usually in larger corporations) do few delivery retries within few seconds and then after many hours... So far, this client is still satisfied with GL (set to 10 minutes) since it reduces spam amount by around 50% (about 3k messages a day). Sometimes, when we have troubles with some servers - they are simply added to WL. It doesn't happen too often, although this is a typical business - with lots of mailing campanies, emails that sound as if it was a typical spam etc. ;) Greetings, -- Jacek Osiecki joshua at ceti.pl GG:3828944 I don't want something I need. I want something I want. From steve.platt at mrc-bsu.cam.ac.uk Tue Jul 24 17:51:55 2012 From: steve.platt at mrc-bsu.cam.ac.uk (Steve Platt) Date: Tue, 24 Jul 2012 15:51:55 +0100 Subject: [Dovecot] deleted-to-trash plugin problems Message-ID: <500EB68B.9020607@mrc-bsu.cam.ac.uk> We have a user who wants to use Outlook with our Dovecot IMAP server but doesn't like the way Outlook handles deletion with IMAP. Ironically she would like Outlook to move the message to her Trash folder, just like Outlook does with "local" folders, I guess. So I enabled the "deleted-to-trash" plugin (v0.3) on our Dovecot 1.2.17 server (built for SPARC, solaris 10, btw). The problem is that it doesn't seem to work; the message doesn't even appear to get marked for delete. The imap log says: "opening Trash succeeded" and then nothing else! I have found that there is a copy of the message in .Trash/tmp/ suggesting that the plugin may be bailing out part way through. Has anyone any idea what's wrong please? I'll attach the output of dovecot -n ... Thanks, Steve -------------- next part -------------- # 1.2.17: /usr/local/etc/dovecot.conf # OS: SunOS 5.10 sun4u base_dir: /var/run/dovecot/ log_path: /var/log/dovecot.log info_log_path: /var/log/dovecot-info.log protocols: imap imaps managesieve listen(default): * listen(imap): * listen(managesieve): *:2000 ssl_ca_file: /etc/ssl/certs/11230895.ca-bundle ssl_cert_file: /etc/ssl/certs/bernstein.crt ssl_key_file: /etc/ssl/private/bernstein.key ssl_cipher_list: TLSv1+HIGH : !SSLv2 : RC4+MEDIUM : !aNULL : !eNULL : !3DES : @STRENGTH verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(managesieve): /usr/local/libexec/dovecot/managesieve-login mail_location: maildir:~/Maildir mail_debug: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(managesieve): /usr/local/libexec/dovecot/managesieve mail_plugins(default): mail_log deleted_to_trash mail_plugins(imap): mail_log deleted_to_trash mail_plugins(managesieve): mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(managesieve): /usr/local/lib/dovecot/managesieve lda: postmaster_address: postmaster at mrc-bsu.cam.ac.uk mail_plugins: sieve mail_plugin_dir: /usr/local/lib/dovecot/lda log_path: /var/log/dovecot-deliver.log info_log_path: /var/log/dovecot-deliver-info.log auth_socket_path: /var/run/dovecot/auth-master auth default: mechanisms: plain login verbose: yes passdb: driver: passwd-file args: /var/dovecot/mrc-bsu/etc/passwd-file userdb: driver: static args: uid=vmail gid=vmail home=/var/dovecot/mrc-bsu/home/%u socket: type: listen client: path: /var/run/dovecot/auth-client mode: 438 master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: mail_log_events: delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields: uid box msgid size subject sieve_dir: ~/sieve deleted_to_trash_folder: Trash From ssilva at sgvwater.com Tue Jul 24 18:38:58 2012 From: ssilva at sgvwater.com (Scott Silva) Date: Tue, 24 Jul 2012 08:38:58 -0700 Subject: [Dovecot] deleted-to-trash plugin problems In-Reply-To: <500EB68B.9020607@mrc-bsu.cam.ac.uk> References: <500EB68B.9020607@mrc-bsu.cam.ac.uk> Message-ID: on 7/24/2012 7:51 AM Steve Platt spake the following: > We have a user who wants to use Outlook with our Dovecot IMAP server but > doesn't like the way Outlook handles deletion with IMAP. > > Ironically she would like Outlook to move the message to her Trash folder, > just like Outlook does with "local" folders, I guess. > > So I enabled the "deleted-to-trash" plugin (v0.3) on our Dovecot 1.2.17 server > (built for SPARC, solaris 10, btw). > > The problem is that it doesn't seem to work; the message doesn't even appear > to get marked for delete. > > The imap log says: "opening Trash succeeded" and then nothing else! > > I have found that there is a copy of the message in .Trash/tmp/ suggesting > that the plugin may be bailing out part way through. > > Has anyone any idea what's wrong please? > > I'll attach the output of dovecot -n ... > > Thanks, > Steve On a side note... Outlook 2010 finally does this on its own... From role.Dovecot-Readers at JLAssocs.com Tue Jul 24 19:59:24 2012 From: role.Dovecot-Readers at JLAssocs.com (J E Lyon) Date: Tue, 24 Jul 2012 17:59:24 +0100 Subject: [Dovecot] deleted-to-trash plugin problems In-Reply-To: <500EB68B.9020607@mrc-bsu.cam.ac.uk> References: <500EB68B.9020607@mrc-bsu.cam.ac.uk> Message-ID: On 24 Jul 2012, at 15:51, Steve Platt wrote: > We have a user who wants to use Outlook with our Dovecot IMAP server but doesn't like the way Outlook handles deletion with IMAP. > > Ironically she would like Outlook to move the message to her Trash folder, just like Outlook does with "local" folders, I guess. > > So I enabled the "deleted-to-trash" plugin (v0.3) on our Dovecot 1.2.17 server (built for SPARC, solaris 10, btw). > > The problem is that it doesn't seem to work; the message doesn't even appear to get marked for delete. > > The imap log says: "opening Trash succeeded" and then nothing else! > > I have found that there is a copy of the message in .Trash/tmp/ suggesting that the plugin may be bailing out part way through. > > Has anyone any idea what's wrong please? You've built from source, I take it? I wanted to use the deleted_to_trash plugin for *exactly* the same reasons as you've cited, but I'm using RPMs on CentOS where most plugins are included as standard, but frustratingly the deleted_to_trash is missing . . I got no response from the list on the subject . . Maybe the plugin isn't bundled in the RPMs that I've seen because it isn't sufficiently "production standard" as it stands, which would fit with your finding it bails part way through the Maildir movement... How very frustrating... And as Scott pointed out, Outlook 2010 has added this functionality -- Microsoft have a bulletin explaining the many IMAP improvements in Outlook 2010, but I have many clients that use older Outlooks and many who find Outlook handling of most things across anything less than a 100% reliable LAN connection to be flaky at times. Which has led me to not push too hard to upgrade to 2010 even if the clients were willing/able. I used to do a lot of C++ coding and the like... but never looked at a project like Dovecot... I wonder how big and/or complicated the plugin is and whether I could debug it and build it for inclusion in the RPMs... How hard could it be, I wonder... Hmmm..... ~ James. From marc at perkel.com Tue Jul 24 20:16:59 2012 From: marc at perkel.com (Marc Perkel) Date: Tue, 24 Jul 2012 10:16:59 -0700 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: <500ED88B.3090506@perkel.com> Try this: http://www.junkemailfilter.com/spam/ On 7/23/2012 8:58 PM, fy wrote: > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > best ? > > > From asai at globalchangemusic.org Tue Jul 24 21:27:03 2012 From: asai at globalchangemusic.org (Asai) Date: Tue, 24 Jul 2012 11:27:03 -0700 Subject: [Dovecot] SSL Warnings in Debug Logs Message-ID: <500EE8F7.6040700@globalchangemusic.org> Greetings, In doing some debugging of authentication issues, I'm wondering if these SSL warnings are anything to be investigating? Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x10, ret=1: before/accept initialization [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: before/accept initialization [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv2/v3 read client hello A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 read client hello A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write server hello A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write certificate A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write key exchange A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write server done A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2002, ret=-1: SSLv3 read client certificate A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 read client key exchange A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 read finished A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write change cipher spec A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 write finished A [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: SSLv3 flush data [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x20, ret=1: SSL negotiation finished successfully [192.168.70.101] Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2002, ret=1: SSL negotiation finished successfully [192.168.70.101] [root at triata ~]# doveconf -n # 2.0.14: /etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.32.1.el5xen x86_64 CentOS release 5.5 (Final) ext3 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = plain mail_debug = yes mail_home = /vmail/%d/%n/home mail_location = maildir:/vmail/%d/%n managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date namespace { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-mysql.conf driver = sql } plugin { mail_log_fields = uid box msgid size from sieve = /vmail/%d/%n/sievescript } protocols = imap pop3 lmtp sieve sieve service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = vmail mode = 0660 user = vmail } unix_listener auth-userdb { mode = 0660 user = vmail } user = root } service imap-login { process_min_avail = 3 service_count = 0 vsz_limit = 0 } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 0 service_count = 1 vsz_limit = 64 M } ssl_cert = References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> <1343133026.5634.6.camel@tardis> <500EA3B0.40705@list-post.mks-mail.de> Message-ID: <1343179530.4455.3.camel@tardis> On Tue, 2012-07-24 at 15:31 +0200, Markus Sch?nhaber wrote: > 24.07.2012 14:30, Noel Butler: > > > On Tue, 2012-07-24 at 14:06 +0200, Markus Sch?nhaber wrote: > > > >> You must not accept mail you are unwilling or unable to deliver - ever! > > > > > > That insisted behaviour was changed four years ago, read up on RFC 5321 > > Where does it say so? > > > IIRC > > I doubt you do. > s6.2 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From noel.butler at ausics.net Wed Jul 25 04:29:25 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 25 Jul 2012 11:29:25 +1000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <20120724134627.GF18334@state-of-mind.de> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> <500E9F09.7030503@univ-nantes.fr> <20120724134627.GF18334@state-of-mind.de> Message-ID: <1343179765.4455.5.camel@tardis> and like all the other constant off-topic crud here, you are free to filter it out if you don't wish to see it. On Tue, 2012-07-24 at 15:46 +0200, Patrick Ben Koetter wrote: > People, > > this is a mailing list dedicated to Dovecot and the protocols POP, IMAP and > MANAGESIEVE with the one or the other detour to storage. > > Greylisting and other Anti-Spam techniques, as discussed in this thread, > truely are off-topic. Please take discussion offlist or to another list that > deals with such stuff. > > p at rick > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From noel.butler at ausics.net Wed Jul 25 04:34:03 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 25 Jul 2012 11:34:03 +1000 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <500ED88B.3090506@perkel.com> References: <500E1D65.7000808@5dshu.com> <500ED88B.3090506@perkel.com> Message-ID: <1343180043.4455.7.camel@tardis> On Tue, 2012-07-24 at 10:16 -0700, Marc Perkel wrote: > Try this: > > http://www.junkemailfilter.com/spam/ > It's also a good idea to place a disclaimer when advertising _your_ products and services on someone else's list > On 7/23/2012 8:58 PM, fy wrote: > > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is > > best ? > > > > > > > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From netwiz at crc.id.au Wed Jul 25 04:56:21 2012 From: netwiz at crc.id.au (Steven Haigh) Date: Wed, 25 Jul 2012 11:56:21 +1000 Subject: [Dovecot] Sieve & Adding headers. Message-ID: <500F5245.2010202@crc.id.au> Hi all, I've just migrated my mail system from procmail to managesieve/sieve and I'm having trouble trying to duplicate a could of rules I used to use in my procmail config. One particular rule would be this: :0 Wfh * ^Sender: owner-scientific-linux-devel at LISTSERV.FNAL.GOV | (sed -e '/^Subject:/ s/\[SCIENTIFIC-LINUX-DEVEL\] *//g'| sed ':a;N;$!ba;s/Precedence: list/Precedence: list\nList-Post: /g') :0 A .Mail\ Lists.SL-Devel/ This in effect adds the header "List-Post: " after "Precedence: list". This allows me to use the Reply To List functionality in Thunderbird. Another rule is: :0 Wfh * ^X-BeenThere: users at lists.repoforge.org | sed -e '/^Subject:/ s/\[users\] *//g' :0 A .Mail\ Lists.RPMForge/ This removes the [users] tag from the subject of each message to the RPMForge / RepoForge mailing lists. Is there a way to do this with sieve? Nothing I have seen seems to do this - however I'm only a beginner in this area. Please CC myself as well as the list. I'm not currently subscribed. -- Steven Haigh Email: netwiz at crc.id.au Web: http://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299 -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4965 bytes Desc: S/MIME Cryptographic Signature URL: From matthew at atom.net Wed Jul 25 05:39:55 2012 From: matthew at atom.net (Matthew Powell) Date: Tue, 24 Jul 2012 22:39:55 -0400 Subject: [Dovecot] fts-lucene vs SEARCH HEADER Message-ID: <293205F7-F702-4124-87F4-D626A988C291@atom.net> Hello. This is in Dovecot 2.1.7 on Linux x86, but it looks as though 2.1.8 has the same issue. I'm using fts-lucene. SEARCH HEADER TO returns no results, but SEARCH TO works: B SEARCH TO TEST * SEARCH 1 2 3 4 5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 24 26 27 28 29 31 B OK Search completed (0.000 secs). C SEARCH HEADER TO TEST * SEARCH C OK Search completed (0.000 secs). Without FTS enabled, both forms of the query work. It looks as though the header names are indexed in lower case. But Dovecot is passing the header name to the backend in upper case when SEARCH HEADER is used. I'm not sure whether this is the *right* fix, but lower-casing the header name in lucene-wrapper.cc seems to fix the problem: --- dovecot-2.1.7-clean/src/plugins/fts-lucene/lucene-wrapper.cc 2011-12-13 06:35:29.000000000 -0500 +++ dovecot-2.1.7/src/plugins/fts-lucene/lucene-wrapper.cc 2012-07-24 14:03:14.000000000 -0400 @@ -1094,7 +1094,7 @@ return false; q = lucene_get_query(index, - t_lucene_utf8_to_tchar(index, arg->hdr_field_name, FALSE), + t_lucene_utf8_to_tchar(index, t_str_lcase(arg->hdr_field_name), FALSE), arg); break; default: Now: B SEARCH TO TEST * SEARCH 1 2 3 4 5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 24 26 27 28 29 31 B OK Search completed (0.000 secs). C SEARCH HEADER TO TEST * SEARCH 1 2 3 4 5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 24 26 27 28 29 31 C OK Search completed (0.000 secs). -- Matthew Powell matthew at atom.net From stephan at rename-it.nl Wed Jul 25 10:16:35 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 25 Jul 2012 09:16:35 +0200 Subject: [Dovecot] Sieve & Adding headers. In-Reply-To: <500F5245.2010202@crc.id.au> References: <500F5245.2010202@crc.id.au> Message-ID: <500F9D53.50206@rename-it.nl> On 7/25/2012 3:56 AM, Steven Haigh wrote: > Hi all, > > I've just migrated my mail system from procmail to managesieve/sieve > and I'm having trouble trying to duplicate a could of rules I used to > use in my procmail config. > > One particular rule would be this: > :0 Wfh > * ^Sender: owner-scientific-linux-devel at LISTSERV.FNAL.GOV > | (sed -e '/^Subject:/ s/\[SCIENTIFIC-LINUX-DEVEL\] *//g'| sed > ':a;N;$!ba;s/Precedence: list/Precedence: list\nList-Post: > /g') > :0 A > .Mail\ Lists.SL-Devel/ > > This in effect adds the header "List-Post: > " after "Precedence: list". > This allows me to use the Reply To List functionality in Thunderbird. > You'll need the editheader extension (meaning a fairly new Pigeonhole v0.3.0+): require ["fileinto", "editheader", "variables", "regex"]; if address "sender" "owner-scientific-linux-devel at LISTSERV.FNAL.GOV" { if header :regex "subject" "\\[SCIENTIFIC-LINUX-DEVEL\\] *(.*)" { deleteheader "Subject"; addheader "Subject" "${1}"; } addheader "List-Post" ""; fileinto "Mail List.SL-Devel"; } > Another rule is: > :0 Wfh > * ^X-BeenThere: users at lists.repoforge.org > | sed -e '/^Subject:/ s/\[users\] *//g' > :0 A > .Mail\ Lists.RPMForge/ > > This removes the [users] tag from the subject of each message to the > RPMForge / RepoForge mailing lists. > > Is there a way to do this with sieve? Nothing I have seen seems to do > this - however I'm only a beginner in this area. > > Please CC myself as well as the list. I'm not currently subscribed. > require ["fileinto", "editheader", "variables", "regex"]; if header "X-BeenThere" "users at lists.repoforge.org" { if header :regex "subject" "\\[users\\] *(.*)" { deleteheader "Subject"; addheader "Subject" "${1}"; } fileinto "Mail Lists.RPMForge"; } Don't forget to enable the editheader extension in your configuration using the sieve_extensions setting. Regards, Stephan. From netwiz at crc.id.au Wed Jul 25 10:57:51 2012 From: netwiz at crc.id.au (Steven Haigh) Date: Wed, 25 Jul 2012 17:57:51 +1000 Subject: [Dovecot] Sieve & Adding headers. In-Reply-To: <500F9D53.50206@rename-it.nl> References: <500F5245.2010202@crc.id.au> <500F9D53.50206@rename-it.nl> Message-ID: <500FA6FF.7070000@crc.id.au> Thanks for the reply Stephan! Out of interest, how can I tell what version is installed? The system is EL6.2 - but dovecot etc is installed via yum. It currently has: # rpm -qa | grep dovecot dovecot-2.0.9-2.el6_1.1.x86_64 dovecot-pigeonhole-2.0.9-2.el6_1.1.x86_64 dovecot-mysql-2.0.9-2.el6_1.1.x86_64 However these look nothing like the pigeonhole version numbers. I can see a few newer packages here and there, but some guidance would be helpful for upgrading either both dovecot & managesieve etc, or just managesieve. -- Steven Haigh Email: netwiz at crc.id.au Web: http://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299 On 25/07/2012 5:16 PM, Stephan Bosch wrote: > On 7/25/2012 3:56 AM, Steven Haigh wrote: >> Hi all, >> >> I've just migrated my mail system from procmail to managesieve/sieve >> and I'm having trouble trying to duplicate a could of rules I used to >> use in my procmail config. >> >> One particular rule would be this: >> :0 Wfh >> * ^Sender: owner-scientific-linux-devel at LISTSERV.FNAL.GOV >> | (sed -e '/^Subject:/ s/\[SCIENTIFIC-LINUX-DEVEL\] *//g'| sed >> ':a;N;$!ba;s/Precedence: list/Precedence: list\nList-Post: >> /g') >> :0 A >> .Mail\ Lists.SL-Devel/ >> >> This in effect adds the header "List-Post: >> " after "Precedence: list". >> This allows me to use the Reply To List functionality in Thunderbird. >> > > You'll need the editheader extension (meaning a fairly new Pigeonhole > v0.3.0+): > > require ["fileinto", "editheader", "variables", "regex"]; > > if address "sender" "owner-scientific-linux-devel at LISTSERV.FNAL.GOV" { > if header :regex "subject" "\\[SCIENTIFIC-LINUX-DEVEL\\] *(.*)" { > deleteheader "Subject"; > addheader "Subject" "${1}"; > } > > addheader "List-Post" ""; > > fileinto "Mail List.SL-Devel"; > } > >> Another rule is: >> :0 Wfh >> * ^X-BeenThere: users at lists.repoforge.org >> | sed -e '/^Subject:/ s/\[users\] *//g' >> :0 A >> .Mail\ Lists.RPMForge/ >> >> This removes the [users] tag from the subject of each message to the >> RPMForge / RepoForge mailing lists. >> >> Is there a way to do this with sieve? Nothing I have seen seems to do >> this - however I'm only a beginner in this area. >> >> Please CC myself as well as the list. I'm not currently subscribed. >> > > require ["fileinto", "editheader", "variables", "regex"]; > > if header "X-BeenThere" "users at lists.repoforge.org" { > if header :regex "subject" "\\[users\\] *(.*)" { > deleteheader "Subject"; > addheader "Subject" "${1}"; > } > fileinto "Mail Lists.RPMForge"; > } > > > Don't forget to enable the editheader extension in your configuration > using the sieve_extensions setting. > > Regards, > > Stephan. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 4965 bytes Desc: S/MIME Cryptographic Signature URL: From stephan at rename-it.nl Wed Jul 25 12:12:37 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Wed, 25 Jul 2012 11:12:37 +0200 Subject: [Dovecot] Sieve & Adding headers. In-Reply-To: <500FA6FF.7070000@crc.id.au> References: <500F5245.2010202@crc.id.au> <500F9D53.50206@rename-it.nl> <500FA6FF.7070000@crc.id.au> Message-ID: <500FB885.7050302@rename-it.nl> Op 7/25/2012 9:57 AM, Steven Haigh schreef: > Thanks for the reply Stephan! > > Out of interest, how can I tell what version is installed? The system > is EL6.2 - but dovecot etc is installed via yum. > > It currently has: > # rpm -qa | grep dovecot > dovecot-2.0.9-2.el6_1.1.x86_64 > dovecot-pigeonhole-2.0.9-2.el6_1.1.x86_64 > dovecot-mysql-2.0.9-2.el6_1.1.x86_64 > > However these look nothing like the pigeonhole version numbers. I can > see a few newer packages here and there, but some guidance would be > helpful for upgrading either both dovecot & managesieve etc, or just > managesieve. I should have mentioned that right away: Pigeonhole 0.3.x is for Dovecot v2.1 Regards, Stephan. From fy at 5dshu.com Wed Jul 25 12:18:24 2012 From: fy at 5dshu.com (fy) Date: Wed, 25 Jul 2012 17:18:24 +0800 Subject: [Dovecot] help about quota config Message-ID: <201207251718242401409@5dshu.com> hi all : mysql mailbox below : CREATE TABLE `mailbox` ( `username` varchar(255) NOT NULL DEFAULT '', `uid` varchar(255) NOT NULL DEFAULT '', `password` varchar(255) NOT NULL DEFAULT '', `clearpwd` varchar(128) NOT NULL DEFAULT '', `name` varchar(255) NOT NULL DEFAULT '', `mailhost` varchar(255) NOT NULL DEFAULT '', `maildir` varchar(255) NOT NULL DEFAULT '', `homedir` varchar(255) NOT NULL DEFAULT '', `quota` varchar(16) NOT NULL DEFAULT '0', `netdiskquota` varchar(16) NOT NULL DEFAULT '0', `domain` varchar(255) NOT NULL DEFAULT '', `uidnumber` int(6) NOT NULL DEFAULT '1000', `gidnumber` int(6) NOT NULL DEFAULT '1000', `createdate` datetime NOT NULL DEFAULT '0000-00-00 00:00:00', `expiredate` date NOT NULL DEFAULT '0000-00-00', `active` smallint(1) NOT NULL DEFAULT '1', `disablepwdchange` smallint(1) DEFAULT NULL, `disablesmtpd` smallint(1) DEFAULT NULL, `disablesmtp` smallint(1) DEFAULT NULL, `disablewebmail` smallint(1) DEFAULT NULL, `disablenetdisk` smallint(1) DEFAULT NULL, `disableimap` smallint(1) DEFAULT NULL, `disablepop3` smallint(1) DEFAULT NULL, `question` text NOT NULL, `answer` text NOT NULL, PRIMARY KEY (`username`), KEY `username` (`username`) ) ENGINE=MyISAM DEFAULT CHARSET=latin1 COMMENT='ExtMail - Virtual Mailboxes' dovecot config below : root at mail:/etc/dovecot# dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-686 i686 Debian 6.0.5 ext3 log_path: /var/log/dovecot.log log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_uid: vuser mail_gid: vgroup mail_location: maildir:/home/data/domains/%d/%n/Maildir mail_debug: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota mail_plugins(imap): quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve managesieve_logout_format(default): bytes=%i/%o managesieve_logout_format(imap): bytes=%i/%o managesieve_logout_format(pop3): bytes=%i/%o managesieve_logout_format(managesieve): bytes(in=%i/out=%o) lda: postmaster_address: fy at 5dshu.com mail_plugins: sieve quota mail_plugin_dir: /usr/lib/dovecot/modules/lda auth_socket_path: /var/run/dovecot/auth-master auth default: mechanisms: plain login verbose: yes debug: yes passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 384 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vuser group: vgroup plugin: quota: dict:user::proxy::quota sieve_dir: /home/data/domains/%d/%n/sieve sieve: /home/data/domains/%d/%n/.dovecot.sieve dict: quota: mysql:/etc/dovecot/dovecot-dict-sql.conf /etc/dovecot/dovecot-dict-sql.conf file : connect = host=localhost dbname=extmail user=root password=bamwei map { pattern = priv/quota/storage table = mailbox username_field = username value_field =quota } map { pattern = priv/quota/messages table = mailbox username_field = username value_field = uid } in the dovecot-sql.conf: user_query= SELECT homedir AS home, maildir, uidnumber AS uid, gidnumber AS gid ,CONCAT( '*:storage=', quota, 'B' ) AS quota_rule FROM mailbox WHERE username ='%u' password_query= SELECT username,domain,password,homedir AS home, maildir, uidnumber AS uid, gidnumber AS gid, CONCAT( '*:storage=', quota, 'B' ) AS userdb_quota_rule FROM mailbox WHERE username ='%u' when is seting quota_rule=*:storage=10M:messages=1000 in plugin quote, all user get 10MB quota. when i comment it .quota no enable . i see the http://wiki1.dovecot.org/Quota/Dict ,but unknow to get Correct setup...thanks ! fy From netwiz at crc.id.au Wed Jul 25 13:00:00 2012 From: netwiz at crc.id.au (Steven Haigh) Date: Wed, 25 Jul 2012 20:00:00 +1000 Subject: [Dovecot] Sieve & Adding headers. In-Reply-To: <500FB885.7050302@rename-it.nl> References: <500F5245.2010202@crc.id.au> <500F9D53.50206@rename-it.nl> <500FA6FF.7070000@crc.id.au> <500FB885.7050302@rename-it.nl> Message-ID: <500FC3A0.5040305@crc.id.au> On 25/07/2012 7:12 PM, Stephan Bosch wrote: > Op 7/25/2012 9:57 AM, Steven Haigh schreef: >> Thanks for the reply Stephan! >> >> Out of interest, how can I tell what version is installed? The system >> is EL6.2 - but dovecot etc is installed via yum. >> >> It currently has: >> # rpm -qa | grep dovecot >> dovecot-2.0.9-2.el6_1.1.x86_64 >> dovecot-pigeonhole-2.0.9-2.el6_1.1.x86_64 >> dovecot-mysql-2.0.9-2.el6_1.1.x86_64 >> >> However these look nothing like the pigeonhole version numbers. I can >> see a few newer packages here and there, but some guidance would be >> helpful for upgrading either both dovecot & managesieve etc, or just >> managesieve. > > I should have mentioned that right away: Pigeonhole 0.3.x is for Dovecot > v2.1 Thanks again! Does anyone on the list happen to know if the atrpms dovecot packages are / have been updated on a regular basis? They seem to be newer than the EL6 ones, but the last build seems to have been 2.1 beta1. -- Steven Haigh Email: netwiz at crc.id.au Web: http://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299 From Ralf.Hildebrandt at charite.de Wed Jul 25 13:43:56 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Jul 2012 12:43:56 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <500DD411.7050107@wildgooses.com> References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> <500DD411.7050107@wildgooses.com> Message-ID: <20120725104355.GI11631@charite.de> * Ed W : > Oh, sorry. Why doveadm though? Why not attack the filesystem directly? It's a bit hard with mdbox: mailboxes. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Jul 25 13:52:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Jul 2012 13:52:24 +0300 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120725104355.GI11631@charite.de> References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> <500DD411.7050107@wildgooses.com> <20120725104355.GI11631@charite.de> Message-ID: On 25.7.2012, at 13.43, Ralf Hildebrandt wrote: > * Ed W : > >> Oh, sorry. Why doveadm though? Why not attack the filesystem directly? > > It's a bit hard with mdbox: mailboxes. Actually it is allowed to simply mv the directory names, but I don't think that's all that much easier than just using doveadm rename. The way I'd do this is to just do doveadm mailbox list, put the strings through some regexps and doveadm rename if necessary. Repeat for all users. From Ralf.Hildebrandt at charite.de Wed Jul 25 13:54:46 2012 From: Ralf.Hildebrandt at charite.de (Ralf Hildebrandt) Date: Wed, 25 Jul 2012 12:54:46 +0200 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> <500DD411.7050107@wildgooses.com> <20120725104355.GI11631@charite.de> Message-ID: <20120725105446.GJ11631@charite.de> * Timo Sirainen : > Actually it is allowed to simply mv the directory names, but I don't > think that's all that much easier than just using doveadm rename. My point exactly. It's such a nice tool, so why not use a tool which keeps the integrity of the mailboxes and folders if it exists. :) > The way I'd do this is to just do doveadm mailbox list, put the strings > through some regexps and doveadm rename if necessary. Repeat for all > users. Yes, something along those lines. It's just that I find it hard to craft a regexp which does that. Maybe after the vacation. -- Ralf Hildebrandt Gesch?ftsbereich IT | Abteilung Netzwerk Charit? - Universit?tsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt at charite.de | http://www.charite.de From tss at iki.fi Wed Jul 25 13:59:51 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Jul 2012 13:59:51 +0300 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <20120725105446.GJ11631@charite.de> References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> <500DD411.7050107@wildgooses.com> <20120725104355.GI11631@charite.de> <20120725105446.GJ11631@charite.de> Message-ID: <4419DC65-DCB2-4595-86E1-1F22CF1895E6@iki.fi> On 25.7.2012, at 13.54, Ralf Hildebrandt wrote: >> The way I'd do this is to just do doveadm mailbox list, put the strings >> through some regexps and doveadm rename if necessary. Repeat for all >> users. > > Yes, something along those lines. It's just that I find it hard to > craft a regexp which does that. Maybe after the vacation. echo " foo/ b a r / baz / sup " | perl -pe 's, +/,/,g; s,/ +,/,g; s/^ +//; s/ +$//' From tss at iki.fi Wed Jul 25 14:31:26 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Jul 2012 14:31:26 +0300 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <20120724134627.GF18334@state-of-mind.de> References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> <500E9F09.7030503@univ-nantes.fr> <20120724134627.GF18334@state-of-mind.de> Message-ID: On 24.7.2012, at 16.46, Patrick Ben Koetter wrote: > this is a mailing list dedicated to Dovecot and the protocols POP, IMAP and > MANAGESIEVE with the one or the other detour to storage. > > Greylisting and other Anti-Spam techniques, as discussed in this thread, > truely are off-topic. Please take discussion offlist or to another list that > deals with such stuff. I think threads like this and storage and maybe others could be moved to some wiki pages. It could be helpful to have a list of possibilities discussing their upsides and downsides, which would work much better in a wiki page than spread into 100 different messages in this list. So, anyone feel free to create http://wiki2.dovecot.org/AntiSpam and start filling it out. From robert at schetterer.org Wed Jul 25 14:35:04 2012 From: robert at schetterer.org (Robert Schetterer) Date: Wed, 25 Jul 2012 13:35:04 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: References: <500E1D65.7000808@5dshu.com> <1343105353.8896.7.camel@tardis> <500E4BD3.2060906@univ-nantes.fr> <7d028932e10b8e74908866444d61ea3b@imt-systems.com> <500E71A5.5000406@univ-nantes.fr> <500E8FBE.5000204@list-post.mks-mail.de> <500E9F09.7030503@univ-nantes.fr> <20120724134627.GF18334@state-of-mind.de> Message-ID: <500FD9E8.5060203@schetterer.org> Am 25.07.2012 13:31, schrieb Timo Sirainen: > On 24.7.2012, at 16.46, Patrick Ben Koetter wrote: > >> this is a mailing list dedicated to Dovecot and the protocols POP, IMAP and >> MANAGESIEVE with the one or the other detour to storage. >> >> Greylisting and other Anti-Spam techniques, as discussed in this thread, >> truely are off-topic. Please take discussion offlist or to another list that >> deals with such stuff. > > I think threads like this and storage and maybe others could be moved to some wiki pages. It could be helpful to have a list of possibilities discussing their upsides and downsides, which would work much better in a wiki page than spread into 100 different messages in this list. > > So, anyone feel free to create http://wiki2.dovecot.org/AntiSpam and start filling it out. > hi Timo, good idea, thx for this by the way , the best anti spam filter , would be the one ,you dont need *g -- Best Regards MfG Robert Schetterer From noel.butler at ausics.net Wed Jul 25 14:47:14 2012 From: noel.butler at ausics.net (Noel Butler) Date: Wed, 25 Jul 2012 21:47:14 +1000 Subject: [Dovecot] mysql auth failover failing In-Reply-To: <4E6E6AE9.6040105@acm.org> References: <4E6ACC6C.6020200@acm.org> <1315830646.7326.46.camel@hurina> <4E6E6AE9.6040105@acm.org> Message-ID: <1343216834.3809.7.camel@tardis> As per this discussion almost a year ago, was there any attempt to introduce failover mode planned Timo? On Mon, 2011-09-12 at 13:26 -0700, Paul B. Henson wrote: > > Perhaps it could be an option, either load balancing between all > available servers, or only using later listed servers when the earlier > listed ones are failing. For my purposes, either way is fine, as long as > authentications don't fail :). The other contributor to this thread, who > has a local mysql replica listed first and the central master listed > second probably wouldn't want the load balanced between them. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From lists at wildgooses.com Wed Jul 25 16:44:34 2012 From: lists at wildgooses.com (Ed W) Date: Wed, 25 Jul 2012 14:44:34 +0100 Subject: [Dovecot] Remove leading and trailing spaces from folder names? In-Reply-To: <4419DC65-DCB2-4595-86E1-1F22CF1895E6@iki.fi> References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> <500DD411.7050107@wildgooses.com> <20120725104355.GI11631@charite.de> <20120725105446.GJ11631@charite.de> <4419DC65-DCB2-4595-86E1-1F22CF1895E6@iki.fi> Message-ID: <500FF842.8080007@wildgooses.com> On 25/07/2012 11:59, Timo Sirainen wrote: > On 25.7.2012, at 13.54, Ralf Hildebrandt wrote: > >>> The way I'd do this is to just do doveadm mailbox list, put the strings >>> through some regexps and doveadm rename if necessary. Repeat for all >>> users. >> Yes, something along those lines. It's just that I find it hard to >> craft a regexp which does that. Maybe after the vacation. > echo " foo/ b a r / baz / sup " | perl -pe 's, +/,/,g; s,/ +,/,g; s/^ +//; s/ +$//' > Bet you can't pronounce all of the above ;-) Stack overflow on doing it in bash (probably would use the perl regexp above though, but only because I understand perl regexps better) http://stackoverflow.com/questions/369758/how-to-trim-whitespace-from-bash-variable Ed W From CMarcus at Media-Brokers.com Wed Jul 25 19:24:22 2012 From: CMarcus at Media-Brokers.com (Charles Marcus) Date: Wed, 25 Jul 2012 12:24:22 -0400 Subject: [Dovecot] OT Re: what best for anti-spam filter? In-Reply-To: References: <500E1D65.7000808@5dshu.com> Message-ID: <50101DB6.4030203@Media-Brokers.com> On 2012-07-24 2:36 AM, Federico Bianchi wrote: > On Tue, 24 Jul 2012, fy wrote: >> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what is >> best ? > If you can afford using a separate "boundary" SMTP (and, thanks to > virtual machines, this is much more common than just a few years ago), > MailAvenger is likely to be a very good solution: for Bayesian filtering > it relies on plain SpamAssassin, but it really shines in doing deep SMTP > transaction analysis, ruling out most spam at that level and making it > far less CPU and memory intensive than its counterparts. > > You can find it at http://mailavenger.org/ Since it doesn't even directly support postfix, I wouldn't even give it a chance. Personally, my dream antispam system would be ASSP integrated with amavisd-new running only as an after-queue content filter, and use postfix's rock-solid built-in pre-queue anti-spam measures. ASSP's Block Reporting feature is really awesome for end users to manage anything in their quarantine. -- Best regards, Charles From me at junc.org Wed Jul 25 22:35:43 2012 From: me at junc.org (Benny Pedersen) Date: Wed, 25 Jul 2012 21:35:43 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: <500E1D65.7000808@5dshu.com> References: <500E1D65.7000808@5dshu.com> Message-ID: Den 2012-07-24 05:58, fy skrev: > what anti-spam for you used ? dspam?spammassian? amavisd-new ? what > is > best ? depends of your gool, if you like to have user controls, then dovecot, dovecot-antispam, with dspam is best, save the spamassassin resources here amavisd-new is NOT an spam filter btw and if you want to have no user control then go for spamassassin currently here i plan to drop spamassassin and only use dspam via dovecot antispam plugin, it save me resources on dns, with lately here is unstable like hell, and talking to dns hosters helps nothing :/ From tofig at freebsd.az Wed Jul 25 22:31:03 2012 From: tofig at freebsd.az (Tofig Suleymanov) Date: Thu, 26 Jul 2012 00:31:03 +0500 Subject: [Dovecot] get recipient entry while in the zlib plugin context Message-ID: <17588E05-C6CB-4102-91CA-B382106C1C81@freebsd.az> Hello, is it possible to get a recipient of an email while being in the zlib plugin context ? Thank you ! -----BEGIN PGP PUBLIC KEY BLOCK----- mQINBE9zuXMBEADG8O1eZpo8gxMFcQ2Bf4K/n2MvDoH5WDnffvLQcswkigKedicz lNNQzMTsk31gH8Hkl3upEK1jd0yfAmB20puITl52Mlzm3Xe6p8JDDZdKBppIaZI+ E0sin1ff/OBn/BPsPQjvCa92kOvm8Km118Wph3mUtQM4kZlMvxWSLWye1l9SUO0b Ou8zbuL0+5YaUABz57BGfJZDMHLORaeJMu3OVty4cpVBZl1Cn6aHNBbTlXK+w6vu 8N0VoDMtoXje2XIuBYhT9e5A104Sl40IU0Uh9K1dwRyyQhqKyUIoBzNMweyWCtN/ qFn2Idp3h48Wca4g2rAnY5jNaihDEL2T/a/aL5jLoLXrzz6bO6Jm1i/NqY932juJ KupcBN6mpMGLStG6/aTw9JC2oggszg2CT8Qu7HdAPweSbbBpH16VCsDgPqfEqOht +ByUUBGjvnVA95MhifUH+ZRaz2LVxgw5s9Hnh3pKVUmx7634MY0YlnS1FTc7ys9E aQWMwwTWGd09qNPMtawwKyMRzkf/m84N8+mnOEGmzMliEPGX+1t5vZ7TBQ+DCvBL HZ4XIKXtQG2of6nbyDbwu9RNWDRyvDCzXqyeoUdqnmTqWyaDyT942uaw59DPx9A7 1+lrAT3iNdBzFnSvYoNwHJ0yCOEbSRYXZKZ9fJClZ9m6ruKEKpC9mmNjFQARAQAB tCNUb2ZpZyBTdWxleW1hbm92IDx0b2ZpZ0BmcmVlYnNkLmF6PokCPwQTAQIAKQUC T3O5cwIbLwUJB4YfgAcLCQgHAwIBBhUIAgkKCwQWAgMBAh4BAheAAAoJEOMBSob+ wYEq8v8P+gJbkfc1sO/AdmWwZDqpyfG+NWQE9nj8gAlngblexX6qowzdBdupZ/1/ BW4qJhZxhY18cVwvF8NdBCTqWDb1iZ1j3zhw69zJdbd3tfqBIRFEeZY48limbIxH 9AcknRKdD+xomjDMbhzIunkYE79Sx3yUAsbB6wqaOvspa1bz9pBEwvGonKJzMBWc 28ieHTNZGaPJurBLlX/ED7N7ZtSflDZhH8wXIWgzdiQrkBRkKrW0J74xkh2EFm9p 82KCnadnukoNGplzdQsj2bODJkpJclz3QvjsyQhwfJdGCJMjgJPoPCT4kwQkABPy IP8tpVHEsZ1UxJG/PsUIV4D1Q8Nypsecx9OrxSdjeBI6JsuuKR2/At9dB2obVNO/ fnhS5WSsNmezrFyKgi3OWcLPwFyItVcVzhzydTvgLIzwH5uJ/w3hFVv45e2ZVReH M05OZCtj4AeTo/WIf8uJCOx1eGMoyy3X7rbzzyt9gSMeuHaKrLIGPZVddI24GZND ZXnlfhZPQvbmj4fjvaQd3Ayy1F8LcziWOT5OmSm12R/GdDKqo0dy646dTKVgka8+ nRGwyMeFGDeMFvZfIURP+efos/KDU8q1KufIHWGt06sGQa8nqakJ555kdlleS9Ol fX08WdgTm4wVEXIc5uXOUm6YVZ/05TZDnHBIIsUug3M0OeGviSSvuQINBE9zuXMB EAC5GRrh23AXpIG6OE8geROcscUhnU7mORAxxfjV6OLN30DLkkmMpGEUz3yRR2ma qRMKyJYtPTpoHqA4j8l7yh+G/ModSg7LToEilGQ8EoKKbSays9s6AtmgSgcLDp8c k8AvlByrMT1lBWIK5sOGlPDbEhYiAkYQbJutQyNRuSSMACFaqKeCqSbp+mJEx2co ODQXD8c8+RA5utL2r5T6NrmTv00s62TEEkTcbHDrXE5PuyxEsa1isnDWK58RjQZE EKjwO0WOlres8PA/EagVNQ6N65auF5jV7eOpgzUu8HCtLLv+OzhVpnTxWAaqlHnd A/B/FS+EtykPq+khE8rFGAWRvYCdmTZhWJg/ku8WAfBCdi9euhNn0t+TjuCam2Rt FquuF0O5+W+8RZLGBdL/NSS66RhyKOW+iO4kVmiUvmTLCFhk0ycsCh8+VgVDtL2y aq6USlJJkZ5F40hkv2JcoNltE5A1B+jbmqT2nLm1qrEEpEhkRJUrINn3GxwrcN3V WzP2tn6fE/0IMTBOAUg8oQ3Okc6IeURINSGkuY2b3SCUSWOmiAB0zL0NaK3vfM92 mKKrceqtRX1AvmOXVVv1ipvP9E5/cwwyTfds4QUaasX3eocF5ZNrAbEX0cCKj3NV C3hGwE/nRjUIsEIp3mJfPcEvppGr1JV3rmgv1UbdF6JvSwARAQABiQREBBgBAgAP BQJPc7lzAhsuBQkHhh+AAikJEOMBSob+wYEqwV0gBBkBAgAGBQJPc7lzAAoJEOf4 16JuYcl6If0P/1ihpZipRHxionQh7454OszRNXtaUa4Wxa8HCnYu6fA5VywerkBP Xnr1Tn70AMNaKiUCc9iLC2L9LdYjXn+lgE2R69AGJGfJnFl3rcvCvCUE0AXV0wBR bdLaWOQ3U++H2U1W9l7E0GcZie8+31IYaQCRjsJsE5w9cWDIaSqEcydkfb/bvp4o Mi8Qj/yQzwOYariilKoIpsdxpwVf1fL9OAk7J/fuyzz+dCypC0kCMjS0ooATkU0s Q7soNh+ZsHNg/Nw91eXsRotwgyErSNK8seoaZzzTLKhdO7SVFta8Ko+ad+KbwiKe n4RHaCssHmf1tgKh2Do+xc9FZz9nXMgf7dCLjD6cExyvVNGjy4ybgRp+C3i3R8e4 CVrRfc6XfjljfQ/2GDH4SVBZ3RGSxW7J4iKhnO7tg4pC1fDdUqriGG6OUrDDuiix xllCdGXlABJQV015wii0LrXtWZPU+oUE/6YmBYCDCHjNRkLW5WfxNzpcSNj8Be5U mGARwmdliXILUsGYglKVSkrk6Pxr6U0WDVSViC28iIo1wvDobPlyZmH0Hw9JXMr6 Rg39j8lrKtxF6x6ElBjrJLmn2XKOXOW6gsjlxlpD/eRxi3YYBDApnTKTd+86GX7X VCLDyEc4ZXYnxEiO5GvSPeiV8LtFVaaefGi3uViMUHMy3FCWRd5Dw/rRagIP/iW9 vRPTdAKY/foYpht/VJGp4N20u25W7oohJbfm/gbPLuKPB/wvQWpt2tf1ImJB2ufG 7GEJ9uPDnEuLxV2cdp0bQOQR4DiqJxXlBeSEeQlxJXVElUztGB/z3Z/CLeCGdVf/ XoCHVdSek+VkZrOY+1VJYo1Q89Z9eAMZAcRyfUUVpcTygy3ay/7Ja+tVvmc3/kHj HUp+OgKKOv170r85MKeeJtm9gakwcuTQTrJ5fTzIUr+DRJyQuxJw5VnF0Gb0VmKq ymjV92Th6sFzoKEGWfBcqbUatlkVR7bGc8tXuk5beIZwVzUwaJiBMLSWXHWO/tFW hKCilLFJ4lewC39ykH0iRDB6dzBSHbcmThpZwG1ZTXHA+AYCd79PmeQimGwKkMGs sgeLEE09ULKdhCML2DhScW6mdM9QqvWfQyA6YMgdI9LsG9aktPOA08lb8SN30P8C 8k2/4DSlIZdwSiKPDlBtfBpA43I4c76DtRZc0knlo+drwZvCZMbu6CCs3gW0ZkZs v5qmkDopG83bysq5ll+pytRRDx2Uzmd6jOXnTE3EFKUjuOUalHIF7NJJIv9lTcon Nb4rYu3KdOkilj06jAQWbHPgk4yI+ZOr95Vu5vbLd2Ft+06Af1UhbIJ5wY2Mkh/q CDhxA3vBoOulljNRUn6iwNSSxAeWekYw+ULUapL0 =wxyH -----END PGP PUBLIC KEY BLOCK----- From h.reindl at thelounge.net Wed Jul 25 22:32:03 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Wed, 25 Jul 2012 21:32:03 +0200 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: References: <500E1D65.7000808@5dshu.com> Message-ID: <501049B3.3000602@thelounge.net> Am 25.07.2012 21:35, schrieb Benny Pedersen: > currently here i plan to drop spamassassin and only use dspam via dovecot antispam plugin, it save me resources on > dns, with lately here is unstable like hell, and talking to dns hosters helps nothing :/ why in the world do you not setup our own DNS? it is not rocket scienece to configure BIND as recursion-resolver and caching server directly on the mail-machine and with high load you should always run local resolvers -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From tss at iki.fi Wed Jul 25 22:46:38 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Jul 2012 22:46:38 +0300 Subject: [Dovecot] what best for anti-spam filter? In-Reply-To: <501049B3.3000602@thelounge.net> References: <500E1D65.7000808@5dshu.com> <501049B3.3000602@thelounge.net> Message-ID: Stop replying here and start writing to http://wiki2.dovecot.org/AntiSpam - I added some kind of a template now. Thread closed. From me at junc.org Wed Jul 25 22:53:13 2012 From: me at junc.org (Benny Pedersen) Date: Wed, 25 Jul 2012 21:53:13 +0200 Subject: [Dovecot] =?utf-8?q?what_best_for_anti-spam_filter=3F?= In-Reply-To: <500E3E98.1080608@filez.com> References: <500E1D65.7000808@5dshu.com> <500E3E98.1080608@filez.com> Message-ID: Den 2012-07-24 08:20, Radim Kolar skrev: >> what anti-spam for you used ? dspam?spammassian? amavisd-new ? what >> is >> best ? > i got best results with dspam + graylist. but dspam is not scalable > solution, it works only if you do not have many users. depends on backend imho i know a hoster that only have dspam filtering pr user level, no complains seen in the forum From rmcgraw at purdue.edu Wed Jul 25 22:50:43 2012 From: rmcgraw at purdue.edu (McGraw, Robert P) Date: Wed, 25 Jul 2012 15:50:43 -0400 Subject: [Dovecot] lock file strangeness Message-ID: We are running dovecot-1.2.10 on a Solaris 10 x86 host. Starting on July 24 I started seeing the following type lock files I the /var/mail directory. -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 agabriel.lock.1343138908.28535.hardy.purdue.edu -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 agabriel.lock.1343138907.28535.hardy.purdue.edu -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 agabriel.lock.1343138906.28535.hardy.purdue.edu -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 agabriel.lock.1343138905.28535.hardy.purdue.edu They are being created by only three of our users. What is common with the three users is that they are using alpine mail client. Again these just started on July 24. We have over 300 user and a lot of them use alpine but only three users seem to generate these lock files. Can some one tell me what and why these are being created and why they are not getting destroyed? Thanks Robert _____________________________________________________________________ Robert P. McGraw, Jr. Manager, Computer System EMAIL: rmcgraw at purdue.edu Purdue University ROOM: MATH-807 Department of Mathematics PHONE: (765) 494-6055 150 N. University Street West Lafayette, IN 47907-2067 From tss at iki.fi Wed Jul 25 23:45:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Wed, 25 Jul 2012 23:45:06 +0300 Subject: [Dovecot] lock file strangeness In-Reply-To: References: Message-ID: Dovecot doesn't create lock files named like these. On 25.7.2012, at 22.50, McGraw, Robert P wrote: > > We are running dovecot-1.2.10 on a Solaris 10 x86 host. > > Starting on July 24 I started seeing the following type lock files I the > /var/mail directory. > > -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 > agabriel.lock.1343138908.28535.hardy.purdue.edu > > -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 > agabriel.lock.1343138907.28535.hardy.purdue.edu > > -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 > agabriel.lock.1343138906.28535.hardy.purdue.edu > > -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 > agabriel.lock.1343138905.28535.hardy.purdue.edu > > They are being created by only three of our users. What is common with the > three > users is that they are using alpine mail client. > > Again these just started on July 24. > > We have over 300 user and a lot of them use alpine but only three users > seem to > generate these lock files. > > Can some one tell me what and why these are being created and why they are > not getting destroyed? > > Thanks > > Robert > > > _____________________________________________________________________ > Robert P. McGraw, Jr. > Manager, Computer System EMAIL: rmcgraw at purdue.edu > Purdue University ROOM: MATH-807 > Department of Mathematics PHONE: (765) 494-6055 > 150 N. University Street > West Lafayette, IN 47907-2067 > > > > > From jtam.home at gmail.com Thu Jul 26 04:16:36 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Wed, 25 Jul 2012 18:16:36 -0700 (PDT) Subject: [Dovecot] lock file strangeness In-Reply-To: References: Message-ID: > From: "McGraw, Robert P" > > We are running dovecot-1.2.10 on a Solaris 10 x86 host. > > Starting on July 24 I started seeing the following type lock files I the > /var/mail directory. > > -rw-rw-rw- 1 agab mail 0 Jul 24 10:08 > agabriel.lock.1343138908.28535.hardy.purdue.edu Hmm, are you sure these are dovecot locks? They look sort of like what alpine would produce. Do your clients have direct filesystem access to the INBOX spool? Joseph Tam From alexandr.sabitov at netregistry.com.au Thu Jul 26 05:27:17 2012 From: alexandr.sabitov at netregistry.com.au (Alexandr Sabitov) Date: Thu, 26 Jul 2012 12:27:17 +1000 Subject: [Dovecot] Problem with running two dovecot instances on the same server Message-ID: <5010AB05.8000709@netregistry.com.au> Hello All, Have anyone met the problem I am struggling with at the moment ? The problem is when I run two instances of dovecot 2.1.8 which are one for proxy-director and another one is a regular one on the same server then I have big amount of strange entries in the /var/log/mail.err: Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs Jul 26 11:52:20 dovecot-test-1 dovecot: director: Error: Empty server list Jul 26 11:52:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting Jul 26 11:52:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs That's for sure set->director_mail_servers variable in main_preinit of ./src/director/main.c But after dumping it it turns out that ppid of the process which failed to use main_preinit function is a "regular" dovecot not the proxy one. When main_preinit is used by dovecot-proxy then set->director_mail_servers = ip address of a mail server and it is all good but if main_preinit is used by dovecot then set->director_mail_servers = empty and I get fatal errors in logs. So here is a question: How come that "non-proxy" dovecot requests director_mail_servers variable and writes fatal error in the logs if it doesn't find it ? Thank you very much for any assistance! Please find my configs: DOVECOT: root at dovecot-test-1:~/software/dovecot/dovecot-2.1.8# /usr/local/dovecot/sbin/dovecot -n # 2.1.8: /usr/local/dovecot/etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_mechanisms = plain digest-md5 cram-md5 apop auth_verbose = yes base_dir = /var/run/dovecot/ default_internal_user = webmail disable_plaintext_auth = no doveadm_password = secret listen = 192.168.0.42 mail_fsync = always mail_location = maildir:~/ mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = " quota fts fts_solr" mmap_disable = yes passdb { args = /usr/local/dovecot/etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { fts = solr fts_solr = break-imap-search url=http://localhost:8080/solr/ quota = maildir:User quota } service doveadm { inet_listener { port = 8282 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service imap { executable = imap } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { executable = pop3 } ssl_cert = References: <20120719124500.GA24108@charite.de> <50081126.9000707@wildgooses.com> <20120719140738.GI24108@charite.de> <500DD411.7050107@wildgooses.com> <20120725104355.GI11631@charite.de> Message-ID: Den 2012-07-25 12:52, Timo Sirainen skrev: > The way I'd do this is to just do doveadm mailbox list, put the > strings through some regexps and doveadm rename if necessary. Repeat > for all users. plans for backports of 2.x to us there still stok on 1.2.x ?, i relly like to get 2.x dovecot going, but it so much diffrent that i keep it on todo :( From netwiz at crc.id.au Thu Jul 26 13:08:57 2012 From: netwiz at crc.id.au (Steven Haigh) Date: Thu, 26 Jul 2012 20:08:57 +1000 Subject: [Dovecot] Sieve & Adding headers. In-Reply-To: <500FB885.7050302@rename-it.nl> References: <500F5245.2010202@crc.id.au> <500F9D53.50206@rename-it.nl> <500FA6FF.7070000@crc.id.au> <500FB885.7050302@rename-it.nl> Message-ID: <50111739.2040208@crc.id.au> On 25/07/2012 7:12 PM, Stephan Bosch wrote: > Op 7/25/2012 9:57 AM, Steven Haigh schreef: >> Thanks for the reply Stephan! >> >> Out of interest, how can I tell what version is installed? The system >> is EL6.2 - but dovecot etc is installed via yum. >> >> It currently has: >> # rpm -qa | grep dovecot >> dovecot-2.0.9-2.el6_1.1.x86_64 >> dovecot-pigeonhole-2.0.9-2.el6_1.1.x86_64 >> dovecot-mysql-2.0.9-2.el6_1.1.x86_64 >> >> However these look nothing like the pigeonhole version numbers. I can >> see a few newer packages here and there, but some guidance would be >> helpful for upgrading either both dovecot & managesieve etc, or just >> managesieve. > > I should have mentioned that right away: Pigeonhole 0.3.x is for Dovecot > v2.1 I'd just like to say a quick thanks to Stephan for his help. I've built 2.1.8 using an updated .spec from the older ATRPMs src.rpm and also built 0.3.1 of dovecot-pigeonhole. The Extensions section of the Server Sieve Capabilities shows editheader now as well as a couple of others. I've made the modifications using the example posted by Stephan earlier in this thread. I'm wondering how the regex works. ie: I've used: if header :regex "subject" "\\[SCIENTIFIC-LINUX-DEVEL\\] *(.*)" Will this also match a subject of "Re: [SCIENTIFIC-LINUX-DEVEL] ...." as well? I'm a bit unsure how the matching will work in this case. -- Steven Haigh Email: netwiz at crc.id.au Web: http://www.crc.id.au Phone: (03) 9001 6090 - 0412 935 897 Fax: (03) 8338 0299 From florian at wagner-flo.net Thu Jul 26 14:26:56 2012 From: florian at wagner-flo.net (Florian Wagner) Date: Thu, 26 Jul 2012 13:26:56 +0200 Subject: [Dovecot] Empty argument in "executable = script ..." service Message-ID: <20120726132656.4f909f6e@auedv3.syscomp.de> Hi, what is the point of adding an empty argument to the command called by the "executable = script ..." setting in a service? I guess that would be the call to array_append_space at line 57 in util/script.c. Background is that I'm trying to run bogofilter through the extprograms Pigeonhole plugin. So I've defined a service entry with a unix_listener and wondered why my test shell script works but bogofilter exits with a cryptic error. Turns out that this empty argument at the end trips up the bogofilter argument parsing code. If I manually call bogofilter with execv and add that empty argument I get the same cryptic error. Regards Florian -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: not available URL: From nicolas at devels.es Thu Jul 26 16:58:12 2012 From: nicolas at devels.es (=?ISO-8859-1?Q?Nicol=E1s?=) Date: Thu, 26 Jul 2012 14:58:12 +0100 Subject: [Dovecot] Auto logging-out client Message-ID: <50114CF4.5090504@devels.es> Hi list! I'm writing because of an issue I'm having with my Postfix-Dovecot installation. Everything's working fine but when an IMAP client is idle for some time, Dovecot automatically logs it out. Jul 26 14:01:16 mail dovecot: imap(nicolas at devels.es): Disconnected: Logged out bytes=1271/7439 This entails errors when trying to send e-mails after this auto logout, such as: host devels.es[91.215.158.237] said: 550-Please turn on SMTP Authentication in your mail client, or login to the 550-IMAP/POP3 server before sending your message. Which result in a very annoying and continuous proccess of restarting my mail client. Is there a way to avoid this auto-logout, or at least configure the time after I want idle clients to auto-logout? Thank you very much! Nicol?s From h.reindl at thelounge.net Thu Jul 26 17:13:46 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 26 Jul 2012 16:13:46 +0200 Subject: [Dovecot] Auto logging-out client In-Reply-To: <50114CF4.5090504@devels.es> References: <50114CF4.5090504@devels.es> Message-ID: <5011509A.7020507@thelounge.net> Am 26.07.2012 15:58, schrieb Nicol?s: > Hi list! > > I'm writing because of an issue I'm having with my Postfix-Dovecot installation. Everything's working fine but when > an IMAP client is idle for some time, Dovecot automatically logs it out. > > Jul 26 14:01:16 mail dovecot: imap(nicolas at devels.es): Disconnected: Logged out bytes=1271/7439 > > This entails errors when trying to send e-mails after this auto logout, such as: > > host devels.es[91.215.158.237] said: > 550-Please turn on SMTP Authentication in your mail client, or login to the > 550-IMAP/POP3 server before sending your message. > > Which result in a very annoying and continuous proccess of restarting my mail client. errors while send messages have really nothing to do with any imap-timeout/logout, these are completly different things! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From h.reindl at thelounge.net Thu Jul 26 17:45:02 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Thu, 26 Jul 2012 16:45:02 +0200 Subject: [Dovecot] Auto logging-out client In-Reply-To: <1ilbr01d5qd0gla9pp1euuyv.1343313712300@email.android.com> References: <1ilbr01d5qd0gla9pp1euuyv.1343313712300@email.android.com> Message-ID: <501157EE.5010605@thelounge.net> FIRST: do not top-post and NOT reply off-list (no i refuse to repair the thread) if he would know that these are different things hewould not write "This entails errors when trying to send e-mails after this auto logout" NO, NO and NO - this entails NOT sending errors! Am 26.07.2012 16:41, schrieb Hans J Albertsson: > Mr Reindl > > Of course Nicolas knows these are different issues! > He wants to know how to avoid the first issue, and merely points out why he desperately wants that fix. > > Skickat fr?n min Samsung Mobil > > Reindl Harald skrev: > > > Am 26.07.2012 15:58, schrieb Nicol?s: >> Hi list! >> >> I'm writing because of an issue I'm having with my Postfix-Dovecot installation. Everything's working fine but when >> an IMAP client is idle for some time, Dovecot automatically logs it out. >> >> Jul 26 14:01:16 mail dovecot: imap(nicolas at devels.es): Disconnected: Logged out bytes=1271/7439 >> >> This entails errors when trying to send e-mails after this auto logout, such as: >> >> host devels.es[91.215.158.237] said: >> 550-Please turn on SMTP Authentication in your mail client, or login to the >> 550-IMAP/POP3 server before sending your message. >> >> Which result in a very annoying and continuous proccess of restarting my mail client. > > errors while send messages have really nothing to do with > any imap-timeout/logout, these are completly different > things! > -- Reindl Harald the lounge interactive design GmbH A-1060 Vienna, Hofm?hlgasse 17 CTO / CISO / Software-Development p: +43 (1) 595 3999 33, m: +43 (676) 40 221 40 icq: 154546673, http://www.thelounge.net/ http://www.thelounge.net/signature.asc.what.htm -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From stephan at rename-it.nl Thu Jul 26 22:34:11 2012 From: stephan at rename-it.nl (Stephan Bosch) Date: Thu, 26 Jul 2012 21:34:11 +0200 Subject: [Dovecot] Sieve & Adding headers. In-Reply-To: <50111739.2040208@crc.id.au> References: <500F5245.2010202@crc.id.au> <500F9D53.50206@rename-it.nl> <500FA6FF.7070000@crc.id.au> <500FB885.7050302@rename-it.nl> <50111739.2040208@crc.id.au> Message-ID: <50119BB3.6070307@rename-it.nl> On 7/26/2012 12:08 PM, Steven Haigh wrote: > On 25/07/2012 7:12 PM, Stephan Bosch wrote: > > I'd just like to say a quick thanks to Stephan for his help. > > I've built 2.1.8 using an updated .spec from the older ATRPMs src.rpm > and also built 0.3.1 of dovecot-pigeonhole. > > The Extensions section of the Server Sieve Capabilities shows > editheader now as well as a couple of others. I've made the > modifications using the example posted by Stephan earlier in this thread. > > I'm wondering how the regex works. ie: I've used: > if header :regex "subject" "\\[SCIENTIFIC-LINUX-DEVEL\\] *(.*)" > > Will this also match a subject of "Re: [SCIENTIFIC-LINUX-DEVEL] ...." > as well? > > I'm a bit unsure how the matching will work in this case. > Well, it will match in that case. But my example will also remove the 'Re:' prefix in that case: there is no '^' at the beginning of the regex, so the regex can match somewhere in the middle of the subject. You can adjust the regex to match a 'Re:' prefix (and whatever else you want to allow) and retain that in a match variable as well: require ["fileinto", "editheader", "variables", "regex"]; if address "sender" "owner-scientific-linux-devel at LISTSERV.FNAL.GOV" { if header :regex "subject" "((Re|Fwd): *)\\[SCIENTIFIC-LINUX-DEVEL\\] *(.*)" { deleteheader "Subject"; addheader "Subject" "${1}${3}"; } addheader "List-Post" ""; fileinto "Mail List.SL-Devel"; } Something like that. Regards, Stephan. From sukhbir_gill at hotmail.com Thu Jul 26 22:38:18 2012 From: sukhbir_gill at hotmail.com (ssukh) Date: Thu, 26 Jul 2012 12:38:18 -0700 (PDT) Subject: [Dovecot] Re move .INBOX from folders. Message-ID: <34217400.post@talk.nabble.com> Hello all, i am just new to dovecot and trying to figure out how to remove .INBOX from the folders. Now all the folders are created like .INBOX.spam .INBOX.temp But i would like to have .spam .temp so on. Result of doveconf -v -------------------------------------------------------------------------------- # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.23.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) auth_username_chars = abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@& auth_verbose = yes default_login_user = dovecot disable_plaintext_auth = no listen = * login_greeting = Dovecot DA ready. mail_access_groups = mail mail_location = maildir:~/Maildir namespace { inbox = yes list = yes location = prefix = separator = . subscriptions = yes type = private } passdb { driver = shadow } passdb { args = username_format=%n /etc/virtual/%d/passwd driver = passwd-file } protocols = imap pop3 service auth { user = root } service imap-login { process_min_avail = 16 user = dovecot } service pop3-login { process_min_avail = 16 user = dovecot } ssl_cert = Hi, We get user's quota limit from LDAP. ?It works fine when user has quota attribute in LDAP (this is when it's different from default quota). But when user doesn't have quota limit attribute Dovecot doesn't take default value defined in quota plugin and assign unlimit quota to user. Our configuration is similar to the example from wiki ( http://wiki2.dovecot.org/Quota/Configuration ). This is the plugin configuration _plugin {_ _? quota = maildir:User quota_ _? quota_rule = *:storage=1G_ _? ?.........._ _}_ And this is the recovered fields in LDAP search: _user_attrs = mailMessageStore=home,mailquotasize=quota_rule=*:bytes=%$_ It works if user has mailquotasize in its LDAP entry. But if it hasn't it Dovecot assign 0 instead of ?*:STORAGE=1G This is the log when LTMP deliver a mail to a user without mailquotasize attribute: _auth: Debug: master in: USER#0111#011eepp68#011service=lmtp#011lip=172.31.100.120#011lport=24#011rip=172.31.100.119#011rport=41452_ _auth: Debug: userdb-cache(eepp68,172.31.100.119): miss_ _auth: Debug: ldap(eepp68,172.31.100.119): user search: base=o=Junta de Castilla-La Mancha,c=ES scope=subtree filter=(uid=eepp68) fields=mailMessageStore,MAILQUOTASIZE_ _auth: Debug: ldap(eepp68,172.31.100.119): result: mailMessageStore=/buzones1/e/eepp68; MAILQUOTASIZE MISSING_ _auth: Debug: master out: USER#0111#011eepp68#011home=/buzones1/e/eepp68#011QUOTA_RULE=*:BYTES=_ _lmtp(8235): Debug: auth input: eepp68 home=/buzones1/e/eepp68 quota_rule=*:bytes=_ _lmtp(8235): Debug: Added userdb setting: plugin/quota_rule=*:bytes=_ _lmtp(8235, eepp68): Debug: Effective uid=11184, gid=2110, home=/buzones1/e/eepp68_ _lmtp(8235, eepp68): Debug: Quota root: name=User quota backend=maildir args=_ _lmtp(8235, eepp68): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0_ I tried it with defining default_fields in userdb but I get the same result: ? ?_userdb {_ ? ? ? ? driver = ldap ? ? ? ? args = /etc/dovecot/dovecot-ldap-lmtp.conf.ext ? ? ? ? default_fields = quota_rule=*:bytes=1073741824 ? ?} If I don't recover mailquotasize (user_attrs = mailMessageStore=home) Dovecot assign default quota limit, so it works in this case: auth: Debug: master in: USER#0111#011eepp68#011service=lmtp#011lip=172.31.100.120#011lport=24#011rip=172.31.100.119#011rport=41427 auth: Debug: userdb-cache(eepp68,172.31.100.119): miss auth: Debug: ldap(eepp68,172.31.100.119): user search: base=o=Junta de Castilla-La Mancha,c=ES scope=subtree filter=(uid=eepp68) fields=mailMessageStore auth: Debug: ldap(eepp68,172.31.100.119): result: mailMessageStore=/buzones1/e/eepp68 auth: Debug: master out: USER#0111#011eepp68#011home=/buzones1/e/eepp68 lmtp(8198): Debug: auth input: eepp68 home=/buzones1/e/eepp68 lmtp(8198, eepp68): Debug: Effective uid=11184, gid=2110, home=/buzones1/e/eepp68 lmtp(8198, eepp68): Debug: Quota root: name=User quota backend=maildir args= lmtp(8198, eepp68): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 Any idea about where is the problem? This is my LDAP configuration for LTMP: hosts = xxx.xxxx.es:389 dn = ........ dnpass = xxxxxxx ldap_version = 3 base = o=mibase,c=ES user_attrs = mailMessageStore=home,mailquotasize=quota_rule=*:bytes=%$ user_filter = (uid=%u) And this is my Dovecot configuracion: # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.1.1.el6.x86_64 x86_64 CentOS release 6.3 (Final)? auth_cache_size = 20 M auth_debug = yes auth_verbose = yes base_dir = /var/run/dovecot/ default_internal_user = vmail default_process_limit = 200 disable_plaintext_auth = no dotlock_use_excl = no hostname = correo.jccm.es lda_mailbox_autocreate = yes login_greeting = Correo jccm.es mail_debug = yes mail_fsync = always mail_gid = 2110 mail_location = maildir:~/Maildir:INDEX=/var/dovecot/index/%n mail_nfs_storage = yes mail_plugins = " quota" mail_uid = 11184 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave vnd.dovecot.pipe vnd.dovecot.execute mbox_write_locks = fcntl mmap_disable = yes namespace inbox { ? inbox = yes ? location =? ? mailbox Drafts { ? ? special_use = \Drafts ? } ? mailbox Junk { ? ? special_use = \Junk ? } ? mailbox Sent { ? ? special_use = \Sent ? } ? mailbox "Sent Messages" { ? ? special_use = \Sent ? } ? mailbox Trash { ? ? special_use = \Trash ? } ? prefix = INBOX. ? separator = . ? type = private } passdb { ? args = /etc/dovecot/dovecot-ldap.conf.ext ? driver = ldap } plugin { ? quota = maildir:User quota ? quota_rule = *:storage=1G ? quota_warning = storage=80%% quota-warning 80 %n ? sieve = ~/.dovecot.sieve ? sieve_execute_bin_dir = /repositorio/paulo/scripts/sieve/extprograms ? sieve_extensions = +vnd.dovecot.execute +vnd.dovecot.pipe ? sieve_filter_bin_dir = /repositorio/paulo/scripts/sieve/extprograms ? sieve_pipe_bin_dir = /repositorio/paulo/scripts/sieve/extprograms ? sieve_plugins = sieve_extprograms } postmaster_address = postmaster at jccm.es protocols = imap pop3 lmtp sieve rejection_reason = Tu mensaje enviado a fue automaticamente rechazado:%n%r rejection_subject = Rechazado: %s service imap-login { ? process_min_avail = 10 ? service_count = 0 } service imap-postlogin { ? drop_priv_before_exec = yes ? executable = script-login /repositorio/paulo/scripts/dovecot/migrar-postlogin.sh ? user = $default_internal_user } service lmtp { ? inet_listener lmtp { ? ? port = 24 ? } } service pop3-login { ? service_count = 0 } service pop3-postlogin { ? drop_priv_before_exec = yes ? executable = script-login /repositorio/paulo/scripts/dovecot/migrar-postlogin.sh ? user = $default_internal_user } service quota-warning { ? executable = script /repositorio/paulo/scripts/postfix-quota-warning.sh ? unix_listener quota-warning { ? ? user = vmail ? } ? user = dovecot } ssl_cert = From daniel.parthey at informatik.tu-chemnitz.de Fri Jul 27 03:00:53 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 27 Jul 2012 02:00:53 +0200 Subject: [Dovecot] Problem with running two dovecot instances on the same server In-Reply-To: <5010AB05.8000709@netregistry.com.au> References: <5010AB05.8000709@netregistry.com.au> Message-ID: <20120727000053.GA15743@daniel.localdomain> Alexandr Sabitov wrote: > The problem is when I run two instances of dovecot 2.1.8 which are > one for proxy-director and another one is a regular one on the same > server then I have big amount of strange entries in the > /var/log/mail.err: > > Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list > Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting > Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs > Jul 26 11:52:20 dovecot-test-1 dovecot: director: Error: Empty server list > Jul 26 11:52:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting > Jul 26 11:52:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs Does behaviour change when you are using more than a single IP address on director_mail_servers variable? Can you elaborate a bit more how the IP addresses are used and on which hosts which IP is UP and running on which interface? > That's for sure set->director_mail_servers variable in main_preinit of ./src/director/main.c > But after dumping it it turns out that ppid of the process which > failed to use main_preinit function is a "regular" dovecot not the > proxy one. > When main_preinit is used by dovecot-proxy then > set->director_mail_servers = ip address of a mail server and it is > all good > but if main_preinit is used by dovecot then > set->director_mail_servers = empty and I get fatal errors in logs. > > So here is a question: How come that "non-proxy" dovecot requests > director_mail_servers variable and writes fatal error in the logs if > it doesn't find it ? Maybe your director is not using the dovecot-proxy.conf? Maybe a conflict between the sockets of the two instances? Please show the output of the following commands: # ps -ef|grep bin/dovecot|grep -v grep # doveadm instance list Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Fri Jul 27 03:34:33 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 27 Jul 2012 02:34:33 +0200 Subject: [Dovecot] Problem overriding default quota limit In-Reply-To: <1343342226.5011c69272387@groupoffice.jccm.es> References: <1343342226.5011c69272387@groupoffice.jccm.es> Message-ID: <20120727003433.GA16019@daniel.localdomain> Hi Raul, Raul Jare?o Morago wrote: > We get user's quota limit from LDAP. ?It works fine when user has > quota attribute in LDAP (this is when it's different from default > quota). But when user doesn't have quota limit attribute Dovecot > doesn't take default value defined in quota plugin and assign unlimit > quota to user. > Our configuration is similar to the example from wiki ( > http://wiki2.dovecot.org/Quota/Configuration ). This is the plugin > configuration > > plugin { > ? quota = maildir:User quota > ? quota_rule = *:storage=1G > } > > And this is the recovered fields in LDAP search: > > user_attrs = > mailMessageStore=home,mailquotasize=quota_rule=*:bytes=%$ > > It works if user has mailquotasize in its LDAP entry. But if it hasn't > it Dovecot assign 0 instead of ?*:STORAGE=1G > > This is the log when LTMP deliver a mail to a user without > mailquotasize attribute: > > _auth: Debug: ldap(eepp68,172.31.100.119): result: > mailMessageStore=/buzones1/e/eepp68; MAILQUOTASIZE MISSING LDAP Entry for mailquotasize is missing... so far so good. > auth: Debug: master out: USER#0111#011eepp68#011home=/buzones1/e/eepp68#011QUOTA_RULE=*:BYTES= > lmtp(8235): Debug: auth input: eepp68 home=/buzones1/e/eepp68 quota_rule=*:bytes= > lmtp(8235): Debug: Added userdb setting: plugin/quota_rule=*:bytes= Here comes the interesting part, ldap "userdb" has returned "QUOTA_RULE=*:BYTES=" and this empty value seems to correspond to 0 Bytes as you can read from the resulting Quota rule: > lmtp(8235, eepp68): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 The "bytes=..." value should be different from 0. http://wiki2.dovecot.org/Quota/Configuration#LDAP states that user_attrs is used only if you use userdb ldap. Where is the "userdb" ldap in your doveconf -n output and where do you get the user information like quota from? > Any idea about where is the problem? The default is not taken since your LDAP query returns a quota_rule with an empty bytes= value when the mailquotasize entry is missing. Regards Daniel -- https://plus.google.com/103021802792276734820 From alexandr.sabitov at netregistry.com.au Fri Jul 27 03:41:27 2012 From: alexandr.sabitov at netregistry.com.au (Alexandr Sabitov) Date: Fri, 27 Jul 2012 10:41:27 +1000 Subject: [Dovecot] Problem with running two dovecot instances on the same server In-Reply-To: <20120727000053.GA15743@daniel.localdomain> References: <5010AB05.8000709@netregistry.com.au> <20120727000053.GA15743@daniel.localdomain> Message-ID: <5011E3B7.802@netregistry.com.au> On 27/07/12 10:00, Daniel Parthey wrote: > Alexandr Sabitov wrote: >> The problem is when I run two instances of dovecot 2.1.8 which are >> one for proxy-director and another one is a regular one on the same >> server then I have big amount of strange entries in the >> /var/log/mail.err: >> >> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list >> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting >> Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs >> Jul 26 11:52:20 dovecot-test-1 dovecot: director: Error: Empty server list >> Jul 26 11:52:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting >> Jul 26 11:52:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs > Does behaviour change when you are using more than a single > IP address on director_mail_servers variable? > > Can you elaborate a bit more how the IP addresses are used > and on which hosts which IP is UP and running on which interface? > >> That's for sure set->director_mail_servers variable in main_preinit of ./src/director/main.c >> But after dumping it it turns out that ppid of the process which >> failed to use main_preinit function is a "regular" dovecot not the >> proxy one. >> When main_preinit is used by dovecot-proxy then >> set->director_mail_servers = ip address of a mail server and it is >> all good >> but if main_preinit is used by dovecot then >> set->director_mail_servers = empty and I get fatal errors in logs. >> >> So here is a question: How come that "non-proxy" dovecot requests >> director_mail_servers variable and writes fatal error in the logs if >> it doesn't find it ? > Maybe your director is not using the dovecot-proxy.conf? > Maybe a conflict between the sockets of the two instances? > > Please show the output of the following commands: > > # ps -ef|grep bin/dovecot|grep -v grep > # doveadm instance list > > Regards > Daniel Hi Daniel and All, Sure, here is an output of the commands: root at dovecot-test-1:/home/asabitov# ps -ef|grep bin/dovecot |grep -v grep root 8214 1 0 Jul26 ? 00:00:04 /usr/local/dovecot/sbin/dovecot -c /usr/local/dovecot/etc/dovecot/dovecot-proxy.conf root 8224 1 0 Jul26 ? 00:00:07 /usr/local/dovecot/sbin/dovecot root at dovecot-test-1:/home/asabitov# /usr/local/dovecot/bin/doveadm instance list path name last used running /var/run/dovecot dovecot 2012-07-26 23:29:41 yes /var/run/dovecot-proxy dovecot-proxy 2012-07-26 23:20:00 yes And the situation changes for the better when I use two separate binaries installed in different folders, so for example this is another server where I do NOT have those fatal errors in the logs: root at dovecot-test-2:/home/asabitov# ps -ef|grep bin/dovecot |grep -v grep root 31859 1 0 Jul25 ? 00:00:10 /usr/local/dovecot/sbin/dovecot root 31871 1 0 Jul25 ? 00:00:03 /usr/local/dovecot-proxy/sbin/dovecot Thank you for your reply anyway. -- Regards, Alexandr Sabitov Systems Administrator Netregistry From alexandr.sabitov at netregistry.com.au Fri Jul 27 03:41:37 2012 From: alexandr.sabitov at netregistry.com.au (Alexandr Sabitov) Date: Fri, 27 Jul 2012 10:41:37 +1000 Subject: [Dovecot] Problem with running two dovecot instances on the same server In-Reply-To: <20120727000053.GA15743@daniel.localdomain> References: <5010AB05.8000709@netregistry.com.au> <20120727000053.GA15743@daniel.localdomain> Message-ID: <5011E3C1.1060809@netregistry.com.au> On 27/07/12 10:00, Daniel Parthey wrote: > Alexandr Sabitov wrote: >> The problem is when I run two instances of dovecot 2.1.8 which are >> one for proxy-director and another one is a regular one on the same >> server then I have big amount of strange entries in the >> /var/log/mail.err: >> >> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list >> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting >> Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs >> Jul 26 11:52:20 dovecot-test-1 dovecot: director: Error: Empty server list >> Jul 26 11:52:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting >> Jul 26 11:52:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs > Does behaviour change when you are using more than a single > IP address on director_mail_servers variable? > > Can you elaborate a bit more how the IP addresses are used > and on which hosts which IP is UP and running on which interface? > >> That's for sure set->director_mail_servers variable in main_preinit of ./src/director/main.c >> But after dumping it it turns out that ppid of the process which >> failed to use main_preinit function is a "regular" dovecot not the >> proxy one. >> When main_preinit is used by dovecot-proxy then >> set->director_mail_servers = ip address of a mail server and it is >> all good >> but if main_preinit is used by dovecot then >> set->director_mail_servers = empty and I get fatal errors in logs. >> >> So here is a question: How come that "non-proxy" dovecot requests >> director_mail_servers variable and writes fatal error in the logs if >> it doesn't find it ? > Maybe your director is not using the dovecot-proxy.conf? > Maybe a conflict between the sockets of the two instances? > > Please show the output of the following commands: > > # ps -ef|grep bin/dovecot|grep -v grep > # doveadm instance list > > Regards > Daniel Hi Daniel and All, Sure, here is an output of the commands: root at dovecot-test-1:/home/asabitov# ps -ef|grep bin/dovecot |grep -v grep root 8214 1 0 Jul26 ? 00:00:04 /usr/local/dovecot/sbin/dovecot -c /usr/local/dovecot/etc/dovecot/dovecot-proxy.conf root 8224 1 0 Jul26 ? 00:00:07 /usr/local/dovecot/sbin/dovecot root at dovecot-test-1:/home/asabitov# /usr/local/dovecot/bin/doveadm instance list path name last used running /var/run/dovecot dovecot 2012-07-26 23:29:41 yes /var/run/dovecot-proxy dovecot-proxy 2012-07-26 23:20:00 yes And the situation changes for the better when I use two separate binaries installed in different folders, so for example this is another server where I do NOT have those fatal errors in the logs: root at dovecot-test-2:/home/asabitov# ps -ef|grep bin/dovecot |grep -v grep root 31859 1 0 Jul25 ? 00:00:10 /usr/local/dovecot/sbin/dovecot root 31871 1 0 Jul25 ? 00:00:03 /usr/local/dovecot-proxy/sbin/dovecot Thank you for your reply anyway. -- Regards, Alexandr Sabitov Systems Administrator Netregistry From dmiller at amfes.com Fri Jul 27 09:01:30 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Thu, 26 Jul 2012 23:01:30 -0700 Subject: [Dovecot] lmtp log message Message-ID: I just tried switching to lmtp from lda. Messages are being delivered, but I now see messages similar to: Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? -- Daniel From shewalkar.rupesh at gmail.com Fri Jul 27 09:35:41 2012 From: shewalkar.rupesh at gmail.com (Rups2603) Date: Thu, 26 Jul 2012 23:35:41 -0700 (PDT) Subject: [Dovecot] Dovecot configuration changes Message-ID: <34219004.post@talk.nabble.com> Dear all, I planning to migrate my Mail server by IRedmail automatic configuration shell script,but my some of Users using pine as Mail user agent.(First They login to Mailserver using SSH with own User Name & Password which is same as mail UserID & PASSWORD & then they do Pine). In IRedmail Their is single mail management user vmail,so How I will change dovecot setting in IRedmail so it will fit in my setup Please see my dovecot configuration http://old.nabble.com/file/p34219004/dovecot.conf dovecot.conf http://old.nabble.com/file/p34219004/dovecot-ldap.conf dovecot-ldap.conf -- View this message in context: http://old.nabble.com/Dovecot-configuration-changes-tp34219004p34219004.html Sent from the Dovecot mailing list archive at Nabble.com. From rjareno at jccm.es Fri Jul 27 10:04:37 2012 From: rjareno at jccm.es (=?utf-8?Q?RAUL_JARE=C3=91O_MORAGO?=) Date: Fri, 27 Jul 2012 09:04:37 +0200 Subject: [Dovecot] Problem overriding default quota limit Message-ID: <1343372677.50123d85d0943@groupoffice.jccm.es> Hi Daniel, The "userdb" ldap is in this section: protocol lmtp { ? mail_plugins = " quota sieve" ? userdb { ? ? args = /etc/dovecot/dovecot-ldap-lmtp.conf.ext ? ? driver = ldap ? } } And dovecot-ldap-lmtp.conf.ex contains this: hosts = xxx.xxx.es:389 dn = uid=xxxxx,o=xxxxxx,c=es dnpass = xxxxxxxxxxx ldap_version = 3 base = o=xxxxxxxxxxxx,c=ES user_attrs = mailMessageStore=home,mailquotasize=quota_rule=*:bytes=%$ user_filter = (uid=%u) A las Viernes, 27-07-2012 en 2:34 Daniel Parthey escribi?: Hi Raul, Raul Jare?o Morago wrote: > We get user's quota limit from LDAP. ?It works fine when user has > quota attribute in LDAP (this is when it's different from default > quota). But when user doesn't have quota limit attribute Dovecot > doesn't take default value defined in quota plugin and assign unlimit > quota to user. > Our configuration is similar to the example from wiki ( > http://wiki2.dovecot.org/Quota/Configuration ). This is the plugin > configuration > > plugin { > ? quota = maildir:User quota > ? quota_rule = *:storage=1G > } > > And this is the recovered fields in LDAP search: > > user_attrs = > mailMessageStore=home,mailquotasize=quota_rule=*:bytes=%$ > > It works if user has mailquotasize in its LDAP entry. But if it hasn't > it Dovecot assign 0 instead of ?*:STORAGE=1G > > This is the log when LTMP deliver a mail to a user without > mailquotasize attribute: > > _auth: Debug: ldap(eepp68,172.31.100.119): result: > mailMessageStore=/buzones1/e/eepp68; MAILQUOTASIZE MISSING LDAP Entry for mailquotasize is missing... so far so good. > auth: Debug: master out: USER#0111#011eepp68#011home=/buzones1/e/eepp68#011QUOTA_RULE=*:BYTES= > lmtp(8235): Debug: auth input: eepp68 home=/buzones1/e/eepp68 quota_rule=*:bytes= > lmtp(8235): Debug: Added userdb setting: plugin/quota_rule=*:bytes= Here comes the interesting part, ldap "userdb" has returned "QUOTA_RULE=*:BYTES=" and this empty value seems to correspond to 0 Bytes as you can read from the resulting Quota rule: > lmtp(8235, eepp68): Debug: Quota rule: root=User quota mailbox=* bytes=0 messages=0 The "bytes=..." value should be different from 0. http://wiki2.dovecot.org/Quota/Configuration#LDAP states that user_attrs is used only if you use userdb ldap. Where is the "userdb" ldap in your doveconf -n output and where do you get the user information like quota from? > Any idea about where is the problem? The default is not taken since your LDAP query returns a quota_rule with an empty bytes= value when the mailquotasize entry is missing. Regards Daniel -- https://plus.google.com/103021802792276734820 From adrian.stoica at dacris.net Fri Jul 27 10:38:31 2012 From: adrian.stoica at dacris.net (Adrian Stoica) Date: Fri, 27 Jul 2012 10:38:31 +0300 Subject: [Dovecot] Question abous ecs Message-ID: Hello, Do you consider using /Envelope/-/Content Splitting/ (ECS) in dovecot ? http://en.wikipedia.org/wiki/Envelope-content_splitting Thanks Adrian Stoica -------------- next part -------------- A non-text attachment was scrubbed... Name: adrian_stoica.vcf Type: text/x-vcard Size: 196 bytes Desc: not available URL: From robert at schetterer.org Fri Jul 27 10:53:16 2012 From: robert at schetterer.org (Robert Schetterer) Date: Fri, 27 Jul 2012 09:53:16 +0200 Subject: [Dovecot] Question abous ecs In-Reply-To: References: Message-ID: <501248EC.3070901@schetterer.org> Am 27.07.2012 09:38, schrieb Adrian Stoica: > Hello, > > Do you consider using /Envelope/-/Content Splitting/ (ECS) in dovecot ? > http://en.wikipedia.org/wiki/Envelope-content_splitting > > Thanks > Adrian Stoica > > The core of ECS was developed in 1996 and U.S. patent 5,815,663 0 was granted in 1998. ? -- Best Regards MfG Robert Schetterer From nicolas at devels.es Fri Jul 27 14:53:16 2012 From: nicolas at devels.es (=?UTF-8?B?Tmljb2zDoXM=?=) Date: Fri, 27 Jul 2012 12:53:16 +0100 Subject: [Dovecot] Auto logging-out client In-Reply-To: <7475559B86C2113B508F0A1A@ritz.innovate.net> References: <50114CF4.5090504@devels.es> <7475559B86C2113B508F0A1A@ritz.innovate.net> Message-ID: <5012812C.1080907@devels.es> El 26/07/2012 15:22, lists-dovecot escribi?: > > ------------ Original Message ------------ >> Date: Thursday, July 26, 2012 02:58:12 PM +0100 >> From: Nicol?s >> To: dovecot at dovecot.org >> Subject: [Dovecot] Auto logging-out client >> >> Hi list! >> >> I'm writing because of an issue I'm having with my Postfix-Dovecot >> installation. Everything's working fine but when an IMAP client is >> idle for some time, Dovecot automatically logs it out. >> >> Jul 26 14:01:16 mail dovecot: imap(nicolas at devels.es): >> Disconnected: Logged out bytes=1271/7439 >> >> This entails errors when trying to send e-mails after this auto >> logout, such as: >> >> host devels.es[91.215.158.237] said: >> 550-Please turn on SMTP Authentication in your mail client, >> or login to the >> 550-IMAP/POP3 server before sending your message. >> >> Which result in a very annoying and continuous proccess of >> restarting my mail client. >> >> Is there a way to avoid this auto-logout, or at least configure >> the time after I want idle clients to auto-logout? >> >> Thank you very much! >> >> Nicol?s >> >> >> > I would suggest that you configure the smtp submission port facility > on your mail server and set your imap/pop clients to send that way. > The "pop before sending" approach is a fairly old and broken hack > (including submitting via port 25, which is frequently blocked by > connectivity providers). > > With sendmail the submission port option is basically configured by > default. I don't know about postfix, but I suspect it is there too. > > > > - Richard > > > Thanks for your responses! I just tried configuring the submission port on postfix but the problem persists even sending by port 587. About what Harald said, I guess I initially missed the point of the problem, it was just my conclusion because that error happens always after the logout message appears, and SMTP authentication is set on my client, so I deduced the problem had something to do with the logout thing. The curious thing is it just happens with SMTP, not with POP3/IMAP protocols. Thanks again! Nicol?s From nicolas at devels.es Fri Jul 27 15:56:37 2012 From: nicolas at devels.es (=?UTF-8?B?Tmljb2zDoXM=?=) Date: Fri, 27 Jul 2012 13:56:37 +0100 Subject: [Dovecot] Auto logging-out client In-Reply-To: <7F52834BE8181BE7283580A7@ritz.innovate.net> References: <50114CF4.5090504@devels.es> <7475559B86C2113B508F0A1A@ritz.innovate.net> <5012812C.1080907@devels.es> <7F52834BE8181BE7283580A7@ritz.innovate.net> Message-ID: <50129005.30405@devels.es> El 27/07/2012 13:37, lists-dovecot escribi?: > > ------------ Original Message ------------ >> Date: Friday, July 27, 2012 12:53:16 PM +0100 >> From: Nicol?s >> To: dovecot at dovecot.org >> Subject: Re: [Dovecot] Auto logging-out client >> >> El 26/07/2012 15:22, lists-dovecot escribi?: >>> ------------ Original Message ------------ >>>> Date: Thursday, July 26, 2012 02:58:12 PM +0100 >>>> From: Nicol?s >>>> To: dovecot at dovecot.org >>>> Subject: [Dovecot] Auto logging-out client >>>> >>>> Hi list! >>>> >>>> I'm writing because of an issue I'm having with my >>>> Postfix-Dovecot installation. Everything's working fine but when >>>> an IMAP client is idle for some time, Dovecot automatically logs >>>> it out. >>>> >>>> Jul 26 14:01:16 mail dovecot: imap(nicolas at devels.es): >>>> Disconnected: Logged out bytes=1271/7439 >>>> >>>> This entails errors when trying to send e-mails after this auto >>>> logout, such as: >>>> >>>> host devels.es[91.215.158.237] said: >>>> 550-Please turn on SMTP Authentication in your mail client, >>>> or login to the >>>> 550-IMAP/POP3 server before sending your message. >>>> >>>> Which result in a very annoying and continuous proccess of >>>> restarting my mail client. >>>> >>>> Is there a way to avoid this auto-logout, or at least configure >>>> the time after I want idle clients to auto-logout? >>>> >>>> Thank you very much! >>>> >>>> Nicol?s >>>> >>>> >>>> >>> I would suggest that you configure the smtp submission port >>> facility on your mail server and set your imap/pop clients to >>> send that way. The "pop before sending" approach is a fairly old >>> and broken hack (including submitting via port 25, which is >>> frequently blocked by connectivity providers). >>> >>> With sendmail the submission port option is basically configured >>> by default. I don't know about postfix, but I suspect it is there >>> too. >>> >>> >>> >>> - Richard >>> >>> >> Thanks for your responses! >> >> I just tried configuring the submission port on postfix but the >> problem persists even sending by port 587. >> >> About what Harald said, I guess I initially missed the point of >> the problem, it was just my conclusion because that error happens >> always after the logout message appears, and SMTP authentication >> is set on my client, so I deduced the problem had something to do >> with the logout thing. The curious thing is it just happens with >> SMTP, not with POP3/IMAP protocols. >> >> Thanks again! >> >> Nicol?s > You indicate that you configured the submission port, but did you > configure the imap/pop client to use it? I.e., explicitly set/change > the send options on the client, including having it use port 587 and > explicitly giving it the id/pw for authenticating against that > service? > > - Richard > > Hi, Yes, I've changed the config on the client too, but now I'm not 100% sure it's working well. I've just configured a client with SMTP port 587, but the line in the log shows still be using port 25: Jul 27 13:53:31 mail postfix/smtp[27598]: B16EF1008D0: to=, relay=*devels.es[91.215.158.237]:25*, delay=0.7, delays=0.11/0.08/0.3/0.21, dsn=2.0.0, status=sent (250 OK id=1Suk2y-0003P3-Tq) Is that the correct behavior? Any additional ideas of why could this problem be happening? Thanks so much for your help! Nicol?s From h.reindl at thelounge.net Fri Jul 27 16:08:22 2012 From: h.reindl at thelounge.net (Reindl Harald) Date: Fri, 27 Jul 2012 15:08:22 +0200 Subject: [Dovecot] Auto logging-out client In-Reply-To: <50129005.30405@devels.es> References: <50114CF4.5090504@devels.es> <7475559B86C2113B508F0A1A@ritz.innovate.net> <5012812C.1080907@devels.es> <7F52834BE8181BE7283580A7@ritz.innovate.net> <50129005.30405@devels.es> Message-ID: <501292C6.7010001@thelounge.net> Am 27.07.2012 14:56, schrieb Nicol?s: > Yes, I've changed the config on the client too, but now I'm not 100% sure it's working well. I've just configured a > client with SMTP port 587, but the line in the log shows still be using port 25: > > Jul 27 13:53:31 mail postfix/smtp[27598]: B16EF1008D0: to=, > relay=*devels.es[91.215.158.237]:25*, delay=0.7, delays=0.11/0.08/0.3/0.21, dsn=2.0.0, status=sent (250 OK > id=1Suk2y-0003P3-Tq) > > Is that the correct behavior? Any additional ideas of why could this problem be happening? this line has NOTHING to do with the client or submission port your POSTFIX is RELAYING a message to "devels.es" which is NEVER submission because submission is for MUA's while 25 is for MTA's and your log show that you have configured relay_host because otherwise postfix would not relay mail to gmail.com over one of your servers you do not see the port a client connects to in maillog finally: this is the wrong mailing-list you have a POSTFIX problem and not one with dovecot! -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 262 bytes Desc: OpenPGP digital signature URL: From joe.beaubien at gmail.com Fri Jul 27 16:41:27 2012 From: joe.beaubien at gmail.com (Joe Beaubien) Date: Fri, 27 Jul 2012 09:41:27 -0400 Subject: [Dovecot] emails are being marked as read auto-magically Message-ID: Hi, I'm having a weird issue with dovecot 2.1.7 (using the latest thunderbird client). Something, I have emails that mark themselves as read on their own. No one touches them, and they get marked as read. I try to re-mark them as unread, but a few seconds later, they get marked as read again. The same problem happens in reverse (emails marks as unread become read). Has anyone ever experienced anything similar? I cannot see any warnings or errors in the dovecot logs, so I'm not sure if the problem is with thunderbird or with dovecot. However, I have never had this issue before switching from dovecot 1.0.X to dovecot 2.1.X. Unfortunately, we updated the thunderbird versions around the same time, so I feel like a dog chasing my tail on this issue. Any help would be appreciated. Regards, -Joe From fcatunda at contactnet.com.br Fri Jul 27 16:44:38 2012 From: fcatunda at contactnet.com.br (Fabio Catunda) Date: Fri, 27 Jul 2012 10:44:38 -0300 Subject: [Dovecot] emails are being marked as read auto-magically In-Reply-To: References: Message-ID: <50129B46.4060208@contactnet.com.br> I really don't know why it happens, but I suggest that you make some tests with some webmail like squirrelmail, it might point you into some direction. Regards, F?bio Catunda. On 27-07-2012 10:41, Joe Beaubien wrote: > Hi, > > I'm having a weird issue with dovecot 2.1.7 (using the latest thunderbird > client). Something, I have emails that mark themselves as read on their > own. No one touches them, and they get marked as read. I try to re-mark > them as unread, but a few seconds later, they get marked as read again. The > same problem happens in reverse (emails marks as unread become read). > > Has anyone ever experienced anything similar? > > I cannot see any warnings or errors in the dovecot logs, so I'm not sure if > the problem is with thunderbird or with dovecot. However, I have never had > this issue before switching from dovecot 1.0.X to dovecot 2.1.X. > Unfortunately, we updated the thunderbird versions around the same time, > so I feel like a dog chasing my tail on this issue. > > Any help would be appreciated. > > Regards, > > -Joe > > From fy at 5dshu.com Fri Jul 27 17:31:37 2012 From: fy at 5dshu.com (fy) Date: Fri, 27 Jul 2012 22:31:37 +0800 Subject: [Dovecot] about dovecot quota ,i will crazy! Message-ID: <201207272231375323271@5dshu.com> Hi ALL: my SQL: user_query=select concat('*bytes=',trim(TRAILING 'S' FROM quota)) as quota_rule,concat('/home/data/domains/',maildir) as mail ,conca t('/home/data/domains/',homedir) as home,uidnumber as uid,gidnumber as gid from mailbox where username='%u' password_query= select username as user ,password,uidnumber as userdb_uid,gidnumber as userdb_gid,concat('*bytes=',trim(TRAILING 'S' FROM quota)) as userdb_quota_rule from mailbox where username='%u' And quota : plugin { quota = maildir:User quota } I got this Error for debug: Jul 27 10:26:13 mail dovecot: imap-login: Login: user=, method=PLAIN, rip=123.65.246.154, lip=123.65.246.151, mpid=11407, session= Jul 27 10:26:13 mail dovecot: imap: Debug: Loading modules from directory: /usr/lib/dovecot/modules Jul 27 10:26:13 mail dovecot: imap: Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so Jul 27 10:26:13 mail dovecot: imap: Debug: Added userdb setting: mail=/home/data/domains/5dshu.com/fy/Maildir/ Jul 27 10:26:13 mail dovecot: imap: Debug: Added userdb setting: plugin/quota_rule=*bytes=52428800 Jul 27 10:26:13 mail dovecot: imap(fy at 5dshu.com): Debug: Effective uid=1000, gid=1000, home=/home/data/domains/5dshu.com/fy Jul 27 10:26:13 mail dovecot: imap(fy at 5dshu.com): Debug: Quota root: name=User quota backend=maildir args= Jul 27 10:26:13 mail dovecot: imap(fy at 5dshu.com): Error: user fy at 5dshu.com: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *bytes=52428800: Invalid rule Jul 27 10:26:13 mail dovecot: imap(fy at 5dshu.com): Error: Invalid user settings. Refer to server log for more information ___________________ Jul 27 10:25:22 mail dovecot: lda: Debug: Added userdb setting: mail=/home/data/domains/5dshu.com/fy/Maildir/ Jul 27 10:25:22 mail dovecot: lda: Debug: Added userdb setting: plugin/quota_rule=*bytes=52428800 Jul 27 10:25:22 mail dovecot: lda(fy at 5dshu.com): Debug: Effective uid=1000, gid=1000, home=/home/data/domains/5dshu.com/fy Jul 27 10:25:22 mail dovecot: lda(fy at 5dshu.com): Debug: Quota root: name=User quota backend=maildir args= Jul 27 10:25:22 mail dovecot: lda(fy at 5dshu.com): Error: user fy at 5dshu.com: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *bytes=52428800: Invalid rule Jul 27 10:25:22 mail dovecot: lda(fy at 5dshu.com): Fatal: Invalid user settings. Refer to server log for more information. Jul 27 10:25:22 mail postfix/pipe[11384]: B3860322100: to=, relay=dovecot, delay=2.3, delays=2.3/0/0/0.03, dsn=4.3.0, status=deferred (temporary failure) What is problem by config?? pls give help , To this question I have been working for a few days . Thank you so much! fy From matthias.lay at securepoint.de Fri Jul 27 17:32:59 2012 From: matthias.lay at securepoint.de (Matthias Lay) Date: Fri, 27 Jul 2012 16:32:59 +0200 Subject: [Dovecot] ntlm auth / usernames with white spaces Message-ID: <1343399579.3072.19.camel@hermann.potsdam.securepoint.local> Hi, we experienced some problems with white spaces in usernames (this was NOT our idea) authenticated bei dovecots ntlmhelper I wrote a little patch fixing this, if you would like. patch applies on 2.1.7 and 2.1.8 (tested those) diff -up ./src/auth/mech-winbind.c.username ./src/auth/mech-winbind.c --- ./src/auth/mech-winbind.c.username 2012-07-27 09:52:33.422553807 +0200 +++ ./src/auth/mech-winbind.c 2012-07-27 14:23:03.718619388 +0200 @@ -239,8 +239,16 @@ do_auth_continue(struct auth_request *au return HR_FAIL; } else if (strcmp(token[0], "AF") == 0) { const char *user, *p, *error; + int ti=1; + + if(gss_spnego) + ti=2; + /* some braindead administrators like spaces in usernames */ + user = token[ti]; + while(token[++ti]){ + user = t_strconcat(user, " ", token[ti], NULL); + } - user = gss_spnego ? token[2] : token[1]; i_assert(user != NULL); p = strchr(user, '\\'); From bob at db.org Fri Jul 27 17:59:56 2012 From: bob at db.org (=?UTF-8?Q?B=C3=A5rd_Johannessen?=) Date: Fri, 27 Jul 2012 16:59:56 +0200 Subject: [Dovecot] about dovecot quota ,i will crazy! In-Reply-To: <201207272231375323271@5dshu.com> References: <201207272231375323271@5dshu.com> Message-ID: <8aab0b446e0586cc9af90d6408142647@db.org> On 2012-07-27 16:31, fy wrote: > my SQL: > user_query=select concat('*bytes=',trim(TRAILING 'S' FROM quota)) as I think that should be '*:bytes=', not '*bytes=' (you're missing a colon). -- Med vennlig hilsen B?rd Johannessen From simon.buongiorno at gmail.com Fri Jul 27 20:01:57 2012 From: simon.buongiorno at gmail.com (Simon Brereton) Date: Fri, 27 Jul 2012 13:01:57 -0400 Subject: [Dovecot] Slightly OT - Winbind and remote authentication. Message-ID: Hi I was doing some research on setting up a samba server for my internal network (as an alternative to NAS). The setup looks easy enough, but then I got to thinking - I already have an internet host with an mysql DB that has usernames and passwords and it would be simpler all round to reuse that as the authentication server. That server is also running Dovecot, so I thought it might be simpler still to set up samba to authenticate using IMAP (the mysql server is not accessible on port 3306 anyway). A quick google resulted in a suggestion from Robert (who by now must be on every single list I'm on or rather vice-versa) about using winbind. I went off to the link he posted - http://wiki.dovecot.org/Authentication/Mechanisms/Winbind?highlight=%28winbind%29 and the problem is that seems to indicate bind and dovecot are on the same server. So, this not being the samba list notwithstanding, is it possible to use a remote dovecot host as an authentication back-end on a local samba installation? Cheers Simon From daniel.parthey at informatik.tu-chemnitz.de Fri Jul 27 23:42:25 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Fri, 27 Jul 2012 22:42:25 +0200 Subject: [Dovecot] lmtp log message In-Reply-To: References: Message-ID: <20120727204225.GA7643@daniel.localdomain> Daniel L. Miller wrote: > I just tried switching to lmtp from lda. Messages are being > delivered, but I now see messages similar to: > > Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: > Client quit (in reset) > > in my log for every message. Is this a normal sign-off - or does > this indicate a misconfiguration? I'm using lmtp and getting a lot of these messages too: Jul 27 07:35:30 10.129.3.213 dovecot: lmtp(22557): Disconnect from 127.0.0.1: Client quit (in reset) Besides this issue http://dovecot.org/list/dovecot/2012-July/066794.html (LMTP timeouts about once a week) I do not have any problems with LMTP delivery. So no need to worry, I would say. Cheers Daniel -- https://plus.google.com/103021802792276734820 From jtam.home at gmail.com Sat Jul 28 01:11:39 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Fri, 27 Jul 2012 15:11:39 -0700 (PDT) Subject: [Dovecot] Dovecot configuration changes In-Reply-To: References: Message-ID: > From: Rups2603 > > I planning to migrate my Mail server by IRedmail automatic > configuration shell script,but my some of Users using pine as Mail user > agent.(First They login to Mailserver using SSH with own User Name & > Password which is same as mail UserID & PASSWORD & then they do Pine). > > In IRedmail Their is single mail management user vmail,so How I > will change dovecot setting in IRedmail so it will fit in my setup >From your brief description, you're going from a virtual user setup to a real user setup, with perhaps direct file access to boot (otherwise, you don't need users to be able to log in to the mail server), in which case, IMAP/POP may be irrelevant. I don't think anybody can give you any concrete idea of how to change your dovecot setup without a lot more information. It appears to me you'll have to do a clean re-do of your dovecot setup, using your old configuration as a reference. Joseph Tam From daniel.parthey at informatik.tu-chemnitz.de Sat Jul 28 02:06:43 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Jul 2012 01:06:43 +0200 Subject: [Dovecot] emails are being marked as read auto-magically In-Reply-To: References: Message-ID: <20120727230643.GA9581@daniel.localdomain> Joe Beaubien wrote: > I'm having a weird issue with dovecot 2.1.7 (using the latest thunderbird > client). Something, I have emails that mark themselves as read on their > own. No one touches them, and they get marked as read. I try to re-mark > them as unread, but a few seconds later, they get marked as read again. The > same problem happens in reverse (emails marks as unread become read). You might enable the mail_log Plugin and log flag changes: http://wiki2.dovecot.org/Plugins/MailLog Something like this should help: mail_plugins = $mail_plugins mail_log notify plugin { mail_log_events = flag_change mail_log_fields = uid box msgid size flags vsize from subject } I also suggest to use some lightweight mail client like "mutt" for testing. Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Jul 28 05:20:52 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Jul 2012 04:20:52 +0200 Subject: [Dovecot] Problem overriding default quota limit In-Reply-To: <1343372677.50123d85d0943@groupoffice.jccm.es> References: <1343372677.50123d85d0943@groupoffice.jccm.es> Message-ID: <20120728022052.GA12150@daniel.localdomain> RAUL JARE?O MORAGO wrote: > And dovecot-ldap-lmtp.conf.ex contains this: > > hosts = xxx.xxx.es:389 > dn = uid=xxxxx,o=xxxxxx,c=es > dnpass = xxxxxxxxxxx > ldap_version = 3 > base = o=xxxxxxxxxxxx,c=ES > user_attrs = mailMessageStore=home,mailquotasize=quota_rule=*:bytes=%$ > user_filter = (uid=%u) It seems that the quota_rule is overridden in any case, even if mailquotasize field is missing. Maybe someone can bring up a fix for this. As a workaround, I recommend you to fill the default mailquotasize value into each LDAP account which has no field "mailquotasize" yet. Regards Daniel -- https://plus.google.com/103021802792276734820 From daniel.parthey at informatik.tu-chemnitz.de Sat Jul 28 05:34:30 2012 From: daniel.parthey at informatik.tu-chemnitz.de (Daniel Parthey) Date: Sat, 28 Jul 2012 04:34:30 +0200 Subject: [Dovecot] Problem overriding default quota limit In-Reply-To: <1343342226.5011c69272387@groupoffice.jccm.es> References: <1343342226.5011c69272387@groupoffice.jccm.es> Message-ID: <20120728023430.GA12958@daniel.localdomain> Hello Raul, Raul Jare?o Morago wrote: > We get user's quota limit from LDAP. ?It works fine when user has > quota attribute in LDAP (this is when it's different from default > quota). But when user doesn't have quota limit attribute Dovecot > doesn't take default value defined in quota plugin and assign unlimit > quota to user. Your version 2.1.1 is affected by a bug which has been fixed in newer Dovecot Releases: http://hg.dovecot.org/dovecot-2.1/rev/4bbc12a87a29 Also see http://dovecot.org/doc/NEWS-2.1 v2.1.7 2012-05-29 Timo Sirainen * LDAP: Compatibility fix for v2.0: ldap: If attributes contain ldapAttr=key=template%$ and ldapAttr doesn't exist, skip the key instead of using "template" value with empty %$ part for the key. Please update to latest stable release 2.1.8 which contains a lot of important fixes. Regards Daniel -- https://plus.google.com/103021802792276734820 From rjareno at jccm.es Sat Jul 28 11:48:48 2012 From: rjareno at jccm.es (=?utf-8?Q?RAUL_JARE=C3=91O_MORAGO?=) Date: Sat, 28 Jul 2012 10:48:48 +0200 Subject: [Dovecot] Problem overriding default quota limit In-Reply-To: <20120728023430.GA12958@daniel.localdomain> References: <20120728023430.GA12958@daniel.localdomain> Message-ID: <1343465328.5013a770449e2@groupoffice.jccm.es> Ok, I'll do it next monday.?Thank you very much!!! A las S?bado, 28-07-2012 en 4:34 Daniel Parthey escribi?: Hello Raul, Raul Jare?o Morago wrote: > We get user's quota limit from LDAP. ?It works fine when user has > quota attribute in LDAP (this is when it's different from default > quota). But when user doesn't have quota limit attribute Dovecot > doesn't take default value defined in quota plugin and assign unlimit > quota to user. Your version 2.1.1 is affected by a bug which has been fixed in newer Dovecot Releases: http://hg.dovecot.org/dovecot-2.1/rev/4bbc12a87a29 Also see http://dovecot.org/doc/NEWS-2.1 v2.1.7 2012-05-29??Timo Sirainen ??* LDAP: Compatibility fix for v2.0: ldap: If attributes contain ????ldapAttr=key=template%$ and ldapAttr doesn't exist, skip the key ????instead of using "template" value with empty %$ part for the key. Please update to latest stable release 2.1.8 which contains a lot of important fixes. Regards Daniel -- https://plus.google.com/103021802792276734820 From tonio at starbridge.org Sat Jul 28 18:21:43 2012 From: tonio at starbridge.org (tonio at starbridge.org) Date: Sat, 28 Jul 2012 17:21:43 +0200 Subject: [Dovecot] fts solr : out of memory Message-ID: <50140387.70805@starbridge.org> Hi Timo, since I've upgraded to 2.1.8, solr indexer doesnt work correctly. I see a lot of these errors in logs: Fatal: master: service(indexer-worker): child 9582 returned error 83 (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may need to increase it)) I've tried to set vsz_limit to a lot more, (around 4096 MB) but same problem occurs. thank for your help ! Regards Tonio Java(TM) SE Runtime Environment (build 1.6.0_31-b04) doveconf -n # 2.1.8: /etc/dovecot/dovecot.conf # OS: Linux 3.2.13-xxxx-std-ipv6-64 x86_64 Debian wheezy/sid xfs auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_mechanisms = plain login dict { quota = mysql:/etc/dovecot/dovecot-dict-quota-sql.conf } hostname = maildir2.spamguard.fr log_timestamp = %Y-%m-%d %H:%M:%S mail_gid = vmail mail_location = sdbox:/home/virtual/%u mail_plugins = quota fts fts_solr acl zlib mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = / type = private } namespace { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX/ separator = / type = private } namespace { list = children location = sdbox:/home/virtual/%%u:INDEX=/home/virtual/%u/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = file:/home/virtual/shared-%d-mailboxes.db autocreate = Spam autocreate2 = SpamFalse autocreate3 = SpamToLearn autocreate4 = Sent autosubscribe = Spam autosubscribe2 = SpamFalse autosubscribe3 = SpamToLearn autosubscribe4 = Sent fts = solr fts_solr = url=http://localhost:8081/solr/ mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename save mailbox_create mail_log_fields = uid box msgid size quota = dict:User quota::proxy::quota quota2 = dict:User quota::file:/home/virtual/%u/dovecot-quota quota_warning = storage=85%% quota-warning 85 %u sieve = ~/dovecot.sieve sieve_before = /home/virtual/sieve/ sieve_dir = ~/sieve zlib_save = bz2 zlib_save_level = 9 } protocols = imap pop3 sieve lmtp service auth { unix_listener /var/spool/postfix-in2/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix-maildir2/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix-msa/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix-smtp06/private/auth { group = mail mode = 0660 user = postfix } unix_listener /var/spool/postfix/private/auth { group = mail mode = 0660 user = postfix } unix_listener auth-in2-master { mode = 0660 user = vmail } unix_listener auth-maildir2-master { mode = 0660 user = vmail } unix_listener auth-master { mode = 0660 user = vmail } unix_listener auth-msa-master { mode = 0660 user = vmail } unix_listener auth-out-master { mode = 0660 user = vmail } unix_listener auth-smtp06-master { mode = 0660 user = vmail } unix_listener auth-userdb { mode = 0660 user = vmail } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 } process_limit = 256 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { address = 46.105.108.210 port = 24 } user = vmail vsz_limit = 368 M } service managesieve-login { inet_listener sieve { port = 4190 } process_limit = 256 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 } process_limit = 256 process_min_avail = 4 service_count = 0 vsz_limit = 256 M } service quota-warning { executable = script /usr/bin/quota-warning.sh unix_listener quota-warning { mode = 0600 user = vmail } user = vmail } ssl = required ssl_ca = References: <50140387.70805@starbridge.org> Message-ID: On 2012-07-28, at 11.21, tonio at starbridge.org wrote: > Fatal: master: service(indexer-worker): child 9582 returned error 83 > (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may > need to increase it)) This looks very much like an issue I've been having with fts-lucene. Indexing works (or at least completes without errors) in 2.1.7, but runs out of memory in 2.1.8. doveadm -v index -u username '*' crashes with: doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory doveadm(username): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x41d31) [0xb7699d31] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x47) [0xb7699e47] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb766c958] -> /usr/lib/dovecot/libdovecot.so.0(+0x5420e) [0xb76ac20e] -> /usr/lib/dovecot/libdovecot.so.0(+0x3e358) [0xb7696358] -> /usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x72) [0xb76967f2] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append_space_unsafe+0x2b) [0xb769691b] -> /usr/lib/dovecot/libdovecot.so.0(charset_to_utf8+0xe8) [0xb768c3f8] -> /usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x3dd) [0xb767fa4d] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x11c) [0xb732881c] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xb256) [0xb732e256] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0xf) [0xb774754f] -> doveadm() [0x805736e] -> doveadm() [0x805422d] -> doveadm() [0x8054aad] -> doveadm(doveadm_mail_try_run+0x12e) [0x8054fae] -> doveadm(main+0x3e4) [0x805bf24] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb7527ca6] -> doveadm() [0x8053c11] The problem occurs repeatably with specific messages. It looks as though the crash happens in lib-charset/charset-iconv.c and the problem was introduced by this change: http://hg.dovecot.org/dovecot-2.1/rev/0fde692cb565. If I remove that one change from 2.1.8, I can index the affected messages again. I've posted an example message that triggers the crash here: https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test case, but I've encountered the same behaviour with a real message. -- Matthew Powell matthew at atom.net From tonio at starbridge.org Sat Jul 28 19:27:59 2012 From: tonio at starbridge.org (tonio at starbridge.org) Date: Sat, 28 Jul 2012 18:27:59 +0200 Subject: [Dovecot] fts solr : out of memory In-Reply-To: References: <50140387.70805@starbridge.org> Message-ID: <5014130F.2060409@starbridge.org> Le 28/07/2012 18:09, Matthew Powell a ?crit : > On 2012-07-28, at 11.21, tonio at starbridge.org wrote: > >> Fatal: master: service(indexer-worker): child 9582 returned error 83 >> (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may >> need to increase it)) > This looks very much like an issue I've been having with fts-lucene. Indexing works (or at least completes without errors) in 2.1.7, but runs out of memory in 2.1.8. > > doveadm -v index -u username '*' crashes with: > > doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory > doveadm(username): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x41d31) [0xb7699d31] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x47) [0xb7699e47] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb766c958] -> /usr/lib/dovecot/libdovecot.so.0(+0x5420e) [0xb76ac20e] -> /usr/lib/dovecot/libdovecot.so.0(+0x3e358) [0xb7696358] -> /usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x72) [0xb76967f2] -> /usr/lib/dovecot/libdovecot.so.0(buffer_append_space_unsafe+0x2b) [0xb769691b] -> /usr/lib/dovecot/libdovecot.so.0(charset_to_utf8+0xe8) [0xb768c3f8] -> /usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x3dd) [0xb767fa4d] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x11c) [0xb732881c] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xb256) [0xb732e256] -> /usr/lib/dovecot/libdovecot-storage.so.0(mail_precache+0xf) [0xb774754f] -> doveadm() [0x805736e] -> doveadm() [0x805422d] -> doveadm() [0x8054aad] -> doveadm(doveadm_mail_try_run+0x12e) [0x8054fae] -> doveadm(main+0x3e4) [0x805bf24] -> /lib/i686/cmov/libc.so.6(__libc_start_main+0xe6) [0xb7527ca6] -> doveadm() [0x8053c11] > > The problem occurs repeatably with specific messages. It looks as though the crash happens in lib-charset/charset-iconv.c and the problem was introduced by this change: http://hg.dovecot.org/dovecot-2.1/rev/0fde692cb565. If I remove that one change from 2.1.8, I can index the affected messages again. > > I've posted an example message that triggers the crash here: https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test case, but I've encountered the same behaviour with a real message. > Hi Matthew, thanks for your answer. most of our email contains French accents, so may be it's related. I've forgot to post the complete log message for this error earlier: Jul 27 19:01:00 mx2 dovecot: indexer-worker(cxx at crxx.fr): Fatal: pool_system_realloc(268435456): Out of memory Jul 27 19:01:00 mx2 dovecot: indexer-worker(cxx at crxx.fr): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43e4a) [0x7fa49c7e4e4a] -> /usr/lib/dovecot/libdovecot.so.0(+0x43e8e) [0x7fa49c7e4e8e] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fa49c7bc784] -> /usr/lib/dovecot/libdovecot.so.0(+0x542f0) [0x7fa49c7f52f0] -> /usr/lib/dovecot/libdovecot.so.0(+0x40fe5) [0x7fa49c7e1fe5] -> /usr/lib/dovecot/libdovecot.so.0(buffer_get_space_unsafe+0x6b) [0x7fa49c7e297b] -> /usr/lib/dovecot/libdovecot.so.0(charset_to_utf8+0xe7) [0x7fa49c7d9867] -> /usr/lib/dovecot/libdovecot.so.0(message_decoder_decode_next_block+0x3b8) [0x7fa49c7ce9d8] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x194) [0x7fa49af6dc34] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc570) [0x7fa49af72570] -> dovecot/indexer-worker() [0x402544] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7fa49c7f1b96] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xa7) [0x7fa49c7f2827] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fa49c7f15a8] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fa49c7de2c3] -> dovecot/indexer-worker(main+0xe2) [0x402002] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7fa49c438ead] -> dovecot/indexer-worker() [0x4020a1] Jul 27 19:01:00 mx2 dovecot: indexer: Error: Indexer worker disconnected, discarding 1 requests for cxx at crxx.fr Jul 27 19:01:00 mx2 dovecot: imap(cxx at crxx.fr): Error: indexer failed to index mailbox INBOX Jul 27 19:01:00 mx2 dovecot: indexer-worker(cxx at crxx): Fatal: master: service(indexer-worker): child 9582 returned error 83 (Out of memory (service indexer-worker { vsz_limit=256 MB }, you may need to increase it)) Hope this helps ! From tss at iki.fi Sat Jul 28 19:34:16 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:34:16 +0300 Subject: [Dovecot] fts solr : out of memory In-Reply-To: References: <50140387.70805@starbridge.org> Message-ID: <43E45374-4BD1-4F45-B389-3EE3D6236A33@iki.fi> On 28.7.2012, at 19.09, Matthew Powell wrote: > This looks very much like an issue I've been having with fts-lucene. Indexing works (or at least completes without errors) in 2.1.7, but runs out of memory in 2.1.8. > > doveadm -v index -u username '*' crashes with: > > doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory .. > I've posted an example message that triggers the crash here: https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test case, but I've encountered the same behaviour with a real message. Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/cbd2b321a68f From tss at iki.fi Sat Jul 28 19:36:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:36:37 +0300 Subject: [Dovecot] lmtp log message In-Reply-To: References: Message-ID: <740C1AC1-AEE2-4872-BCAD-FB89A92A4E60@iki.fi> On 27.7.2012, at 9.01, Daniel L. Miller wrote: > I just tried switching to lmtp from lda. Messages are being delivered, but I now see messages similar to: > > Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) > > in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? It's normal. From tss at iki.fi Sat Jul 28 19:39:11 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:39:11 +0300 Subject: [Dovecot] Slightly OT - Winbind and remote authentication. In-Reply-To: References: Message-ID: <036E6265-AC79-416E-926C-7E8E36DD5F15@iki.fi> On 27.7.2012, at 20.01, Simon Brereton wrote: > I was doing some research on setting up a samba server for my internal > network (as an alternative to NAS). The setup looks easy enough, but > then I got to thinking - I already have an internet host with an mysql > DB that has usernames and passwords and it would be simpler all round > to reuse that as the authentication server. That server is also > running Dovecot, so I thought it might be simpler still to set up > samba to authenticate using IMAP (the mysql server is not accessible > on port 3306 anyway). A quick google resulted in a suggestion from > Robert (who by now must be on every single list I'm on or rather > vice-versa) about using winbind. I went off to the link he posted - > http://wiki.dovecot.org/Authentication/Mechanisms/Winbind?highlight=%28winbind%29 > and the problem is that seems to indicate bind and dovecot are on the > same server. That link is about Dovecot authenticating via Samba, not vice versa. > So, this not being the samba list notwithstanding, is it possible to > use a remote dovecot host as an authentication back-end on a local > samba installation? No idea, but I think at least some Samba auth mechanisms probably want to have access to the plaintext password. From tss at iki.fi Sat Jul 28 19:44:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:44:28 +0300 Subject: [Dovecot] Auto logging-out client In-Reply-To: <50114CF4.5090504@devels.es> References: <50114CF4.5090504@devels.es> Message-ID: <47BFE4EB-05F5-49D8-A3EC-5CA8592A8D57@iki.fi> On 26.7.2012, at 16.58, Nicol?s wrote: > I'm writing because of an issue I'm having with my Postfix-Dovecot installation. Everything's working fine but when an IMAP client is idle for some time, Dovecot automatically logs it out. > > Jul 26 14:01:16 mail dovecot: imap(nicolas at devels.es): Disconnected: Logged out bytes=1271/7439 IMAP client logs out. Dovecot doesn't automatically do anything. There's no way (even theoretically) to prevent this from Dovecot's side. > This entails errors when trying to send e-mails after this auto logout, such as: > > host devels.es[91.215.158.237] said: > 550-Please turn on SMTP Authentication in your mail client, or login to the > 550-IMAP/POP3 server before sending your message. Your server supports pop3/imap-before-smtp kludge. I'd remove that entirely and require SMTP AUTH always. From tss at iki.fi Sat Jul 28 19:49:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:49:20 +0300 Subject: [Dovecot] Problem with running two dovecot instances on the same server In-Reply-To: <5010AB05.8000709@netregistry.com.au> References: <5010AB05.8000709@netregistry.com.au> Message-ID: <1F7239B6-9E6D-4CB8-8B51-BB9CDF000199@iki.fi> On 26.7.2012, at 5.27, Alexandr Sabitov wrote: > Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list > Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting > Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs Director process gets started for the backend installation where it of course isn't configured. My guess: doveadm (from some cronjob?) is attempting to connect to /var/run/dovecot/director-admin instead of /var/run/dovecot-proxy/director-admin. From tss at iki.fi Sat Jul 28 19:50:07 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:50:07 +0300 Subject: [Dovecot] Re move .INBOX from folders. In-Reply-To: <34217400.post@talk.nabble.com> References: <34217400.post@talk.nabble.com> Message-ID: <454E10F5-07CB-4B50-A168-C7931E101355@iki.fi> On 26.7.2012, at 22.38, ssukh wrote: > > Hello all, i am just new to dovecot and trying to figure out how to remove > .INBOX from the folders. > > Now all the folders are created like > .INBOX.spam > .INBOX.temp > > But i would like to have > .spam > .temp Either: a) Remove INBOX. namespace prefix from your client's configuration. or b) Set prefix=INBOX. to Dovecot's namespace configuration From tss at iki.fi Sat Jul 28 19:53:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:53:22 +0300 Subject: [Dovecot] mysql auth failover failing In-Reply-To: <1343216834.3809.7.camel@tardis> References: <4E6ACC6C.6020200@acm.org> <1315830646.7326.46.camel@hurina> <4E6E6AE9.6040105@acm.org> <1343216834.3809.7.camel@tardis> Message-ID: <6A2EDD80-CCE6-4445-91A9-9FA4349DFE1E@iki.fi> It's in my TODO, but I don't know when I'll get around to implementing it. So many things to do right now.. On 25.7.2012, at 14.47, Noel Butler wrote: > As per this discussion almost a year ago, was there any attempt to > introduce failover mode planned Timo? > > > On Mon, 2011-09-12 at 13:26 -0700, Paul B. Henson wrote: > > >> >> Perhaps it could be an option, either load balancing between all >> available servers, or only using later listed servers when the earlier >> listed ones are failing. For my purposes, either way is fine, as long as >> authentications don't fail :). The other contributor to this thread, who >> has a local mysql replica listed first and the central master listed >> second probably wouldn't want the load balanced between them. > > From tss at iki.fi Sat Jul 28 19:57:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 19:57:30 +0300 Subject: [Dovecot] SSL Warnings in Debug Logs In-Reply-To: <500EE8F7.6040700@globalchangemusic.org> References: <500EE8F7.6040700@globalchangemusic.org> Message-ID: <24CBCC03-620C-4D49-AB71-ADCCA3836579@iki.fi> On 24.7.2012, at 21.27, Asai wrote: > Greetings, > > In doing some debugging of authentication issues, I'm wondering if these SSL warnings are anything to be investigating? > > Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x10, ret=1: before/accept initialization [192.168.70.101] > Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: before/accept initialization [192.168.70.101] They should be debug messages, not warnings. Fixed in hg. From tss at iki.fi Sat Jul 28 20:01:06 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:01:06 +0300 Subject: [Dovecot] fts-lucene vs SEARCH HEADER In-Reply-To: <293205F7-F702-4124-87F4-D626A988C291@atom.net> References: <293205F7-F702-4124-87F4-D626A988C291@atom.net> Message-ID: On 25.7.2012, at 5.39, Matthew Powell wrote: > It looks as though the header names are indexed in lower case. But Dovecot is passing the header name to the backend in upper case when SEARCH HEADER is used. > > I'm not sure whether this is the *right* fix, but lower-casing the header name in lucene-wrapper.cc seems to fix the problem: Committed with another change: http://hg.dovecot.org/dovecot-2.1/rev/77f2510bb009 From tss at iki.fi Sat Jul 28 20:02:28 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:02:28 +0300 Subject: [Dovecot] get recipient entry while in the zlib plugin context In-Reply-To: <17588E05-C6CB-4102-91CA-B382106C1C81@freebsd.az> References: <17588E05-C6CB-4102-91CA-B382106C1C81@freebsd.az> Message-ID: On 25.7.2012, at 22.31, Tofig Suleymanov wrote: > is it possible to get a recipient of an email while being in the zlib plugin context ? Maybe. Depends on where exactly you want to get it. The recipient of the email would anyway be the struct mail_user->username however you can get it. From tss at iki.fi Sat Jul 28 20:04:29 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:04:29 +0300 Subject: [Dovecot] maildir_copy_with_hardlinks on v.2.0.19 In-Reply-To: References: Message-ID: <58254FC4-5787-4FD1-8C15-36E20D860E44@iki.fi> On 23.7.2012, at 22.12, mailing list subscriber wrote: > As requested, here is my update. As you can see I am running now the > latest release however emails delivered through lmtp gets split into > different files instead of expected hardlinked files. .. > userdb { > driver = passwd > } Looks like you're using system users. Each mail then needs to be written using different permissions, so hard linking can't work. From tss at iki.fi Sat Jul 28 20:08:20 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:08:20 +0300 Subject: [Dovecot] Dsync replication In-Reply-To: <5006E334.7010903@airstreamcomm.net> References: <4FF5C09F.2060001@airstreamcomm.net> <213B51F00051AE48A9F0E11288017717B8402D@Delta.sc.local> <4FF74A76.6010605@airstreamcomm.net> <43111E61-9EA3-4D18-B55B-C0F40DD891BE@iki.fi> <5006E334.7010903@airstreamcomm.net> Message-ID: <49AA27A4-89AE-4F9E-96E8-CFB0E29964AD@iki.fi> On 18.7.2012, at 19.24, list at airstreamcomm.net wrote: >> Anyway, in my small installation I'm using it in active-active mode and it works well enough. I've even configured my clients intentionally so that they use different servers. > > Does dsync replication only work between two hosts? In my scenario I would have two sites with X number of nodes at each with an NFS backend for each site. For this example lets say I have site A with two nodes that mount one NFS share, and site B with two nodes that mount one NFS share. Is it possible to implement dsync replication between these two clusters of nodes? You can have as many hosts as you want, but you most likely don't want to use active-active setup via NFS or you'll run into NFS caching (= corruption) problems. From tss at iki.fi Sat Jul 28 20:10:30 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:10:30 +0300 Subject: [Dovecot] Disable W= in filenames In-Reply-To: <5E24DAB287DDB44CA15AE0C4C7D2B5D0107DDBDB@MentDCEX01.red007.local> References: <5E24DAB287DDB44CA15AE0C4C7D2B5D0107DDBDB@MentDCEX01.red007.local> Message-ID: <1333956F-1B84-4C3E-8787-E6EA265F2610@iki.fi> On 19.7.2012, at 21.18, Daniel Pieper wrote: > I implemented a plugin for dovecot and now i got a error that maildir filename has wrong W value. When i rename the filename manually and delete the W value, all is ok. Maybe your plugin should fix also the value? It sounds like you would also run into trouble with dovecot.index.cache file. See zlib plugin as an example how to fix the W value ("virtual size"). I think you can also disable it that way by setting it to (uoff_t)-1. From tss at iki.fi Sat Jul 28 20:14:43 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:14:43 +0300 Subject: [Dovecot] [PATCH] Pop3/imap ordering in courier-dovecot-migrate.pl In-Reply-To: <5006CCB8.6050701@websupport.sk> References: <5006CCB8.6050701@websupport.sk> Message-ID: <48EB25C0-028A-40DC-9333-37A664EF3373@iki.fi> On 18.7.2012, at 17.48, Pavel Stano wrote: > current version of courier migration script courier-dovecot-migrate.pl preserve order of POP3 UIDLs, but it keep IMAP UIDs not in order which cause problems with clients using IMAP along with POP3 in some cases. > > Attached patch use MAIL_FETCH_POP3_ORDER feature added in version 2.0.13 to keep POP3 and IMAP UID[L]s in order. Applied. I had wanted to add that feature for a while now but never had time myself. From tss at iki.fi Sat Jul 28 20:34:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 20:34:35 +0300 Subject: [Dovecot] expire-tool stops on first box with nothing to expire in it In-Reply-To: References: Message-ID: <0FB06895-C9CA-46A3-A6E2-92518A63EC50@iki.fi> On 19.7.2012, at 4.24, Joseph Tam wrote: > I previously observed that the "date.saved" field (tested by savedbefore) > seems to be clustered around particular timestamps. > > Dumping out this field using > > doveadm -ftab fetch -A \ > "mailbox date.saved" \ > \( mailbox Trash OR mailbox Junk OR mailbox "Deleted Items" \) > > it seems the timestamps do not actually correspond to when the client > moves messages to their trash folder, but rather, to the time I query > for it i.e. a query instantiates this value. This timestamp is then > retained. > > In practice, if this query is done on a periodic interval (e.g. daily), > the date.saved will be roughly correct, and is usable as an age test > for expungement. > > Is there a way to force this value to be cached when the client actually > saves or moves a message? Done for save: http://hg.dovecot.org/dovecot-2.1/rev/c598f76eeeed Fixed copy: http://hg.dovecot.org/dovecot-2.1/rev/b13b8267999d It still doesn't add it to cache if it's not already cached in both the source and destination mailbox, but usually it should be after initial doveadm run. From bob at db.org Sat Jul 28 21:50:51 2012 From: bob at db.org (=?UTF-8?Q?B=C3=A5rd_Johannessen?=) Date: Sat, 28 Jul 2012 20:50:51 +0200 Subject: [Dovecot] lmtp log message In-Reply-To: <740C1AC1-AEE2-4872-BCAD-FB89A92A4E60@iki.fi> References: <740C1AC1-AEE2-4872-BCAD-FB89A92A4E60@iki.fi> Message-ID: <340fce1afab7ddaa03f45655f2edafb2@db.org> On 2012-07-28 18:36, Timo Sirainen wrote: > On 27.7.2012, at 9.01, Daniel L. Miller wrote: >> Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: >> Client quit (in reset) >> in my log for every message. Is this a normal sign-off - or does >> this indicate a misconfiguration? > > It's normal. I'm not even sure it's noteworthy enough to justify logging it; if the message is going to be exactly the same in all (non error) cases, what's the point of logging it? -- Med vennlig hilsen B?rd Johannessen From tss at iki.fi Sat Jul 28 22:01:32 2012 From: tss at iki.fi (Timo Sirainen) Date: Sat, 28 Jul 2012 22:01:32 +0300 Subject: [Dovecot] lmtp log message In-Reply-To: <340fce1afab7ddaa03f45655f2edafb2@db.org> References: <740C1AC1-AEE2-4872-BCAD-FB89A92A4E60@iki.fi> <340fce1afab7ddaa03f45655f2edafb2@db.org> Message-ID: <988AB312-2D5A-411B-8759-FCD31184CAC6@iki.fi> On 28.7.2012, at 21.50, B?rd Johannessen wrote: > On 2012-07-28 18:36, Timo Sirainen wrote: >> On 27.7.2012, at 9.01, Daniel L. Miller wrote: >>> Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) >>> in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? >> >> It's normal. > > I'm not even sure it's noteworthy enough to justify logging it; if the message is going to be exactly the same in all (non error) cases, what's the point of logging it? Since LMTP is supposed to be accessible only internally from your network, maybe there isn't much of a point.. Maybe if DATA command is used it wouldn't be logged, but otherwise a QUIT would log "Client quit without delivering mail". Would be useful to get some opinions from more people though. From tonio at starbridge.org Sun Jul 29 02:20:32 2012 From: tonio at starbridge.org (tonio at starbridge.org) Date: Sun, 29 Jul 2012 01:20:32 +0200 Subject: [Dovecot] fts solr : out of memory In-Reply-To: <43E45374-4BD1-4F45-B389-3EE3D6236A33@iki.fi> References: <50140387.70805@starbridge.org> <43E45374-4BD1-4F45-B389-3EE3D6236A33@iki.fi> Message-ID: <501473C0.2080106@starbridge.org> Le 28/07/2012 18:34, Timo Sirainen a ?crit : > On 28.7.2012, at 19.09, Matthew Powell wrote: > >> This looks very much like an issue I've been having with fts-lucene. Indexing works (or at least completes without errors) in 2.1.7, but runs out of memory in 2.1.8. >> >> doveadm -v index -u username '*' crashes with: >> >> doveadm(username): Fatal: pool_system_realloc(536870912): Out of memory > .. >> I've posted an example message that triggers the crash here: https://www.atom.net/dovecot/testcase1.eml. That's a very contrived test case, but I've encountered the same behaviour with a real message. > Thanks, fixed: http://hg.dovecot.org/dovecot-2.1/rev/cbd2b321a68f > > Thanks Timo, it seems ok now with memory problem, but an other error shows up: doveadm -v index -u clxx at spamguard.fr INBOX doveadm(clxx at spamguard.fr): Info: INBOX: Caching mails seq=1..59363 21500/59363doveadm(clxx at spamguard.fr): Error: fts_solr: Invalid XML input at line 1: mismatched tag doveadm(clxx at spamguard.fr): Panic: file solr-connection.c: line 545 (solr_connection_post_more): assertion failed: (maxfd >= 0) doveadm(clxx at spamguard.fr): Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(+0x43d9a) [0x7f0fbaa87d9a] -> /usr/lib/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7f0fbaa87e5a] -> /usr/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f0fbaa5f62a] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(solr_connection_post_more+0x249) [0x7f0fb8bef899] -> /usr/lib/dovecot/modules/lib21_fts_solr_plugin.so(+0x4a87) [0x7f0fb8beca87] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(fts_build_mail+0x339) [0x7f0fb9210dd9] -> /usr/lib/dovecot/modules/lib20_fts_plugin.so(+0xc570) [0x7f0fb9215570] -> doveadm() [0x412ce1] -> doveadm() [0x40fbd2] -> doveadm() [0x4105db] -> doveadm(doveadm_mail_try_run+0x161) [0x410a41] -> doveadm(main+0x375) [0x40f7e5] -> /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xfd) [0x7f0fba6dbead] -> doveadm() [0x40f9d1] Aborted It doesnt seems happened too often, but I've some mailboxes that cant be indexed From dmiller at amfes.com Sun Jul 29 02:50:51 2012 From: dmiller at amfes.com (Daniel L. Miller) Date: Sat, 28 Jul 2012 16:50:51 -0700 Subject: [Dovecot] lmtp log message In-Reply-To: <988AB312-2D5A-411B-8759-FCD31184CAC6@iki.fi> References: <50122EBA.8020208@amfes.com> <740C1AC1-AEE2-4872-BCAD-FB89A92A4E60@iki.fi> <340fce1afab7ddaa03f45655f2edafb2@db.org> <988AB312-2D5A-411B-8759-FCD31184CAC6@iki.fi> Message-ID: On 7/28/2012 12:01 PM, Timo Sirainen wrote: > On 28.7.2012, at 21.50, B?rd Johannessen wrote: > >> On 2012-07-28 18:36, Timo Sirainen wrote: >>> On 27.7.2012, at 9.01, Daniel L. Miller wrote: >>>> Jul 26 22:59:21 bubba dovecot: lmtp(20508): Disconnect from local: Client quit (in reset) >>>> in my log for every message. Is this a normal sign-off - or does this indicate a misconfiguration? >>> It's normal. >> I'm not even sure it's noteworthy enough to justify logging it; if the message is going to be exactly the same in all (non error) cases, what's the point of logging it? > Since LMTP is supposed to be accessible only internally from your network, maybe there isn't much of a point.. Maybe if DATA command is used it wouldn't be logged, but otherwise a QUIT would log "Client quit without delivering mail". Would be useful to get some opinions from more people though. > To clarify - I see this message after EVERY message reception. My last log section is: Jul 28 16:21:03 bubba assp/smtpd[27486]: connect from localhost[127.0.0.1] Jul 28 16:21:04 bubba assp/smtpd[27486]: EBBE1404015C: client=dovecot.org[193.210.130.67] Jul 28 16:21:05 bubba postfix/cleanup[27490]: EBBE1404015C: message-id=<501473C0.2080106 at starbridge.org> Jul 28 16:21:05 bubba postfix/qmgr[20352]: EBBE1404015C: from=, size=6944, nrcpt=1 (queue active) Jul 28 16:21:05 bubba dovecot: lmtp(26702): Connect from local Jul 28 16:21:05 bubba dovecot: lmtp(26702, dmiller at amfes.com): 16WjK+FzFFBOaAAA4TDHRA: sieve: msgid=<501473C0.2080106 at starbridge.org>: stored mail into mailbox 'Lists/Dovecot-Users' Jul 28 16:21:05 bubba postfix/lmtp[27536]: EBBE1404015C: to=, relay=mail.amfes.com[private/dovecot-lmtp], delay=1.2, delays=0.99/0.01/0/0.2, dsn=2.0.0, status=sent (250 2.0.0 16WjK+FzFFBOaAAA4TDHRA Saved) Jul 28 16:21:05 bubba assp/smtpd[27486]: disconnect from dovecot.org[193.210.130.67] Jul 28 16:21:05 bubba dovecot: lmtp(26702): Disconnect from local: Client quit (in reset) Jul 28 16:21:05 bubba postfix/qmgr[20352]: EBBE1404015C: removed Everything looks clean - except that lmtp disconnect message. It may not be an error - or a warning - but it looks like one. If I need to change my configuration in either Postfix or Dovecot to "fix" it - please tell me so. Otherwise, if the message just read "Disconnect from local" to match with the initial connect it would be fine. But as it is - it leaves me feeling I've got something setup wrong. And indeed - do I? In Postfix, I've simply set my transport to use the lmtp via unix socket. I have NOT set a destination_recipient_limit - as I didn't think I needed to do so, unlike when use the lda via pipe. I had lmtp timeouts, so I set process_min_avail=5. -- Daniel From tss at iki.fi Sun Jul 29 03:10:01 2012 From: tss at iki.fi (Timo Sirainen) Date: Sun, 29 Jul 2012 03:10:01 +0300 Subject: [Dovecot] lmtp log message In-Reply-To: References: <50122EBA.8020208@amfes.com> <740C1AC1-AEE2-4872-BCAD-FB89A92A4E60@iki.fi> <340fce1afab7ddaa03f45655f2edafb2@db.org> <988AB312-2D5A-411B-8759-FCD31184CAC6@iki.fi> Message-ID: On 29.7.2012, at 2.50, Daniel L. Miller wrote: > Jul 28 16:21:05 bubba dovecot: lmtp(26702): Disconnect from local: Client quit (in reset) > > Everything looks clean - except that lmtp disconnect message. It may not be an error - or a warning - but it looks like one. If I need to change my configuration in either Postfix or Dovecot to "fix" it - please tell me so. Otherwise, if the message just read "Disconnect from local" to match with the initial connect it would be fine. But as it is - it leaves me feeling I've got something setup wrong. It simply says that the LMTP client sent a QUIT command. Nothing wrong. Maybe it would be less bad looking without the "(in reset)" text? From peter at grimmond.org Sun Jul 29 01:14:21 2012 From: peter at grimmond.org (Peter Grimmond) Date: Sat, 28 Jul 2012 22:14:21 +0000 Subject: [Dovecot] Problem with preserving MIME format with Dovecot IMAP and Outlook Message-ID: Hello, I have installed Dovecot 2.0.19 on Ubuntu 12.04 LTS. I have set up a very basic IMAP configuration. I have two different laptops running Outlook (one MacOS, one Windows 7) and I have configured both to connect to the Dovecot service. My problem is, emails in MIME format that I synchronise from one Outlook client are not being correctly decoded as MIME messages on the other client. Instead, the message is rendered in plaintext, starting with the lines: This message is in MIME format. Since your mail reader does not understandthis format, some or all of this message may not be legible. --B_3426360453_22814Content-type: multipart/alternative; boundary="B_3426360453_4099" --B_3426360453_4099Content-type: text/plain; charset="ISO-8859-1"Content-transfer-encoding: quoted-printable Can anyone help me understand why this might be happening? The output from dovecot -n is: # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-26-generic-pae i686 Ubuntu 12.04 LTS auth_debug = yes auth_verbose = yes disable_plaintext_auth = no mail_location = maildir:~/Maildir passdb { driver = pam } protocols = " imap" ssl_cert = References: <4E6ACC6C.6020200@acm.org> <1315830646.7326.46.camel@hurina> <4E6E6AE9.6040105@acm.org> <1343216834.3809.7.camel@tardis> <6A2EDD80-CCE6-4445-91A9-9FA4349DFE1E@iki.fi> Message-ID: <1343533514.4297.3.camel@tardis> On Sat, 2012-07-28 at 19:53 +0300, Timo Sirainen wrote: > It's in my TODO, but I don't know when I'll get around to implementing it. So many things to do right now.. > Thanks -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: From dmalolepszy at optusnet.com.au Sun Jul 29 07:30:44 2012 From: dmalolepszy at optusnet.com.au (Dominic Malolepszy) Date: Sun, 29 Jul 2012 14:30:44 +1000 Subject: [Dovecot] Dovecot Director behaviour when two or more fail Message-ID: <5014BC74.10006@optusnet.com.au> Hi, On the Dovecot wiki here http://wiki2.dovecot.org/Director it says the following: "The directors are going to connect to each others in a ring. For example if you have servers called A, B and C, director will create connections A->B, B->C and C->A" I am curious what happens in a scenario when two or more directors fail and potentially break a ring in half, do the active directors detect this and reform a new ring? Dominic From punit_j at rediffmail.com Sun Jul 29 08:30:36 2012 From: punit_j at rediffmail.com (punit_j) Date: 29 Jul 2012 05:30:36 -0000 Subject: [Dovecot] =?utf-8?q?Enable_ACL_for_master_user?= Message-ID: <20120729053036.14847.qmail@f5mail-224-131.rediffmail.com> Hi Team, I have dovecot setup and I have enabled master User login in dovecot.conf file as below :- auth_master_user_separator = * passdb { driver = passwd-file args = /etc/dovecot/passwd.masterusers master = yes pass = yes } passdb { driver = shadow } userdb { driver = passwd } and craeted file. I can login to the user but what I see after login is : - Trying 0.0.0.0... Connected to 0. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED I18NLEVEL=1 STARTTLS AUTH=PLAIN] Dovecot ready. a login user*masteruser password a OK Logged in. a list "" * * LIST (\HasNoChildren) "." "Trash" * LIST (\Unmarked) "." "INBOX" a OK List completed. However when I login as a normal user I see this : - Connected to 0. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 SASL-IR SORT THREAD=REFERENCES MULTIAPPEND UNSELECT LITERAL+ IDLE CHILDREN NAMESPACE LOGIN-REFERRALS UIDPLUS LIST-EXTENDED I18NLEVEL=1 STARTTLS AUTH=PLAIN] Dovecot ready. a login user userpassword a OK Logged in. a list "" * * LIST (\HasNoChildren) "." "Drafts" * LIST (\HasNoChildren) "." "Trash" * LIST (\HasNoChildren) "." "Sent items" * LIST (\HasNoChildren) "." "INBOX" a OK List completed. I think its because of ACL for user and it is using masterusers ACL. I looked at http://wiki2.dovecot.org/PostLoginScripting, however do we have sample login script so that I can use that script with modifications. Sorry I am new to dovecot. Thanks and Regards. From fy at 5dshu.com Sun Jul 29 08:46:23 2012 From: fy at 5dshu.com (fy) Date: Sun, 29 Jul 2012 13:46:23 +0800 Subject: [Dovecot] =?gb2312?b?u9i4tDogUmU6ICBteXNxbCBhdXRoIGZhaWxvdmVyIGZh?= =?gb2312?b?aWxpbmc=?= References: <4E6ACC6C.6020200@acm.org> <1315830646.7326.46.camel@hurina> <4E6E6AE9.6040105@acm.org> <1343216834.3809.7.camel@tardis> <6A2EDD80-CCE6-4445-91A9-9FA4349DFE1E@iki.fi>, <1343533514.4297.3.camel@tardis> Message-ID: <201207291346228504614@5dshu.com> it is so easy to use . if you using dovecot2.0 ,in /etc/dovecot/dovecot-sql.conf.ext like this : connect =host=localhost dbname=extmail user=root password=******** user_query=select concat('*:bytes=',trim(TRAILING 'S' FROM quota)) as quota_rule,concat('/home/data/domains/',maildir) as mail ,concat('/home/data/domains/',homedir) as home,uidnumber as uid,gidnumber as gid from mailbox where username='%u' password_query= select username as user ,password,uidnumber as userdb_uid,gidnumber as userdb_gid,concat('*:bytes=',trim(TRAILING 'S' FROM quota)) as userdb_quota_rule from mailbox where username='%u' my profile include quota, see this http://wiki2.dovecot.org/AuthDatabase/SQL fy From: Noel Butler Date: 2012-07-29 11:45 To: dovecot Subject: Re: [Dovecot] mysql auth failover failing On Sat, 2012-07-28 at 19:53 +0300, Timo Sirainen wrote: > It's in my TODO, but I don't know when I'll get around to implementing it. So many things to do right now.. > Thanks From kurt.armistead at eeb5.eu Sun Jul 29 10:01:47 2012 From: kurt.armistead at eeb5.eu (Armistead, Kurt) Date: Sun, 29 Jul 2012 09:01:47 +0200 Subject: [Dovecot] Address extensions Message-ID: Hi guys, I'm trying to set up in-mailbox routing for messages using the + delimiter. The server runs postfix and dovecot 1.2.15. I have re-defined the delivery service in master.cf as "-f ${sender} -d ${user}@${nexthop} -n -m INBOX.${extension}" (server was formerly courier) but the routing only works when the destination folder has no uppercase characters. "Folder1" doesn't deliver correctly while "folder2" does. What am I doing wrong? Kurt From hendy at soluvas.com Sun Jul 29 15:00:49 2012 From: hendy at soluvas.com (Hendy Irawan) Date: Sun, 29 Jul 2012 05:00:49 -0700 (PDT) Subject: [Dovecot] BUG: LDAP extension failed to authenticate if 'base' DN configuration is empty '' Message-ID: <34226738.post@talk.nabble.com> Dear Dovecot developers, We have several context entries and want to authenticate a user against the whole LDAP server. However when setting "base = " Dovecot LDAP fails with following message: Jul 29 10:50:59 nitik1 dovecot: auth: Error: ldap(hendy at staging.member.berbatik.com,127.0.0.1): ldap_search((virtualMail=hendy at staging.member.berbatik.com)) failed: Other (e.g., implementation specific) error This limits the search to only one base DN, e.g. base = dc=staging,dc=berbatik,dc=com The ldap client library is able to search entries using an empty base DN, so it should be possible for Dovecot LDAP to do the same. An even more elegant solution will be the ability to specify multiple base DNs, but if Dovecot can search the "root DN" (empty string) it would be great. Thank you. Hendy ----- http://www.Soluvas.com/ Soluvas - Making eCommerce Work for You -- View this message in context: http://old.nabble.com/BUG%3A-LDAP-extension-failed-to-authenticate-if-%27base%27-DN-configuration-is-empty-%27%27-tp34226738p34226738.html Sent from the Dovecot mailing list archive at Nabble.com. From asai at globalchangemusic.org Sun Jul 29 20:18:09 2012 From: asai at globalchangemusic.org (Asai) Date: Sun, 29 Jul 2012 10:18:09 -0700 Subject: [Dovecot] SSL Warnings in Debug Logs In-Reply-To: <24CBCC03-620C-4D49-AB71-ADCCA3836579@iki.fi> References: <500EE8F7.6040700@globalchangemusic.org> <24CBCC03-620C-4D49-AB71-ADCCA3836579@iki.fi> Message-ID: <50157051.4020506@globalchangemusic.org> Thank you, Timo. On 07/28/2012 09:57 AM, Timo Sirainen wrote: > On 24.7.2012, at 21.27, Asai wrote: > >> Greetings, >> >> In doing some debugging of authentication issues, I'm wondering if these SSL warnings are anything to be investigating? >> >> Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x10, ret=1: before/accept initialization [192.168.70.101] >> Jul 24 11:23:16 triata dovecot: imap-login: Warning: SSL: where=0x2001, ret=1: before/accept initialization [192.168.70.101] > They should be debug messages, not warnings. Fixed in hg. > > From aajaxx at gmail.com Sun Jul 29 22:09:28 2012 From: aajaxx at gmail.com (Ajax) Date: Sun, 29 Jul 2012 15:09:28 -0400 Subject: [Dovecot] Help needed with configuration giving dict error No such file Message-ID: Each time a mail message is delivered to a mailbox records like the following appear in /var/log/dovecot.warn: Jul 29 10:02:05 iris dovecot: lmtp(29320, jax at XXX.com): Error: file dict commit: file_dotlock_open(/var/vmail/XXX.com/jax/home/maildir/dovecot-quota) failed: No such file or directory Jul 29 10:02:05 iris dovecot: lmtp(29320, jax at XXX.com): Error: file dict commit: file_dotlock_open(/var/vmail/XXX.com/jax/home/maildir/dovecot-quota) failed: No such file or directory Jul 29 10:02:05 iris dovecot: lmtp(29320, jax at XXX.com): Error: dict quota: Quota update failed, it's now desynced This is with Dovecot 2.1.7 installed by Debian Squeeze binary packages, dovecot-core v 1:2.1.7-2~bpo60+1, and friends. The host is running GNU/Linux 3.2.0-0.bpo.2-amd64 and on a xfs filesystem. Entries in /etc/dovecot/users look like this: jax at XXX.com:{plain}password The output from 'doveconf -n' is in the attachment to this mail message. I'm a beginner without experience with either IMAP servers or clients. Can you give me pointers to see what I'm missing that causes the errors logged as above? The user data base looks like this: userdb { args = uid=5000 gid=5000 home=/var/vmail/%d/%n/home mail=maildir:/var/vmail/%d/%n driver = static } Being a beginner, I wonder if is necessary to manually make home directories for each '/var/vmail/%d/%n/home'? I've tried test accounts with and without empty "home" directories at the indicated locations without seeing any difference in the error indications. Is it necessary to manually populate the "home" directories? Besides having no clue about the errors logged, I don't understand what it takes to make quotas work nor do I understand what options are available or advisable. The distribution wiki/QuotaMaildir.txt seems clear that the Maildif+++ backend is indicated but beyond that I remain in the dark. FWIW, I've yet to try a real IMAP client. Thank you for any pointers you can give me. -------------- next part -------------- $ doveconf -n # IP address redacted below # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-0.bpo.2-amd64 x86_64 Debian 6.0.5 xfs auth_debug = yes auth_verbose = yes disable_plaintext_auth = no lmtp_save_to_detail_mailbox = yes mail_debug = yes mail_gid = 5000 mail_location = maildir:/var/vmail/%d/%n mail_plugins = acl quota mail_privileged_group = vmail mail_uid = 5000 namespace inbox { inbox = yes location = prefix = } passdb { args = scheme=plain username_format=%u /etc/dovecot/users driver = passwd-file } plugin { acl = vfile:/var/vmail/conf.d/%d/acls:cache_secs=300 quota = dict:user::file:%h/maildir/dovecot-quota quota_rule = *:storage=1GB quota_rule2 = Trash:storage=+10%% sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_global_dir = /var/vmail/conf.d/%d/sieve } protocols = imap lmtp service auth-worker { user = doveauth } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = doveauth } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = 999.999.999.999 port = 143 } inet_listener imaps { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = References: <5010AB05.8000709@netregistry.com.au> <1F7239B6-9E6D-4CB8-8B51-BB9CDF000199@iki.fi> Message-ID: <501636C0.5070305@netregistry.com.au> On 29/07/12 02:49, Timo Sirainen wrote: > On 26.7.2012, at 5.27, Alexandr Sabitov wrote: >> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list >> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting >> Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs > Director process gets started for the backend installation where it of course isn't configured. My guess: doveadm (from some cronjob?) is attempting to connect to /var/run/dovecot/director-admin instead of /var/run/dovecot-proxy/director-admin. > Hi Timo, Thanks for reply. That's right: strace shows that director process is a child of non-proxy dovecot (pid 7089) which is should not be happening. And I don't have any cronjob running. Might we need just update dovecot wiki that if someone runs dovecot and dovecot-proxy then they have to be installed in different folders as separate binaries otherwise one instance cross over another somehow magically. :) 7089 epoll_wait(51, {}, 56, 25070) = 0 7089 epoll_ctl(51, EPOLL_CTL_ADD, 38, {EPOLLIN|EPOLLPRI|EPOLLERR|EPOLLHUP, {u32=15790944, u64=15790944}}) = 0 7089 epoll_wait(51, {{EPOLLIN, {u32=15790944, u64=15790944}}}, 56, 43627051) = 1 7089 clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f56462d19d0) = 7169 7089 epoll_ctl(51, EPOLL_CTL_DEL, 38, {0, {u32=15790944, u64=15790944}}) = 0 7089 epoll_wait(51, 7169 dup2(83, 2) = 2 7169 close(6) = 0 7169 dup2(38, 7) = 7 7169 dup2(10, 3) = 3 7169 dup2(129, 5) = 5 7169 dup2(95, 6) = 6 7169 setrlimit(RLIMIT_DATA, {rlim_cur=262144*1024, rlim_max=262144*1024}) = 0 7169 setrlimit(RLIMIT_AS, {rlim_cur=262144*1024, rlim_max=262144*1024}) = 0 7169 execve("/usr/local/dovecot/libexec/dovecot/director", ["dovecot/director"], [/* 15 vars */]) = 0 7169 brk(0) = 0xff6000 7169 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) root at dovecot-test-1:/home/asabitov# lsof /var/run/dovecot/director-admin COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME dovecot 7089 root 38u unix 0xffff88007e8e1200 0t0 48243 /var/run/dovecot/director-admin root at dovecot-test-1:/home/asabitov# lsof /var/run/dovecot-proxy/director-admin COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME dovecot 7102 root 39u unix 0xffff88007e9da300 0t0 48469 /var/run/dovecot-proxy/director-admin director 7178 webmail 8u unix 0xffff88007e9da300 0t0 48469 /var/run/dovecot-proxy/director-admin #ps auxf root 7089 0.0 0.0 17332 1092 ? Ss Jul27 0:02 /usr/local/dovecot/sbin/dovecot webmail 7091 0.0 0.0 12808 1100 ? S Jul27 0:00 \_ dovecot/anvil root 7092 0.0 0.0 13056 1324 ? S Jul27 0:01 \_ dovecot/log root 3572 0.0 0.1 14256 2220 ? S Jul29 0:01 \_ dovecot/config webmail 16207 0.0 0.2 71684 4252 ? S 09:11 0:00 \_ dovecot/imap root 7102 0.0 0.0 17204 1072 ? Ss Jul27 0:01 /usr/local/dovecot/sbin/dovecot -c /usr/local/dovecot/etc/dovecot/dovecot-proxy.conf webmail 7104 0.0 0.0 12808 1084 ? S Jul27 0:00 \_ dovecot-proxy/anvil root 7105 0.0 0.0 12936 1244 ? S Jul27 0:00 \_ dovecot-proxy/log webmail 7178 0.0 0.0 12964 1328 ? S Jul27 0:00 \_ dovecot-proxy/director dovenull 16202 0.0 0.1 23500 2916 ? S 09:11 0:00 \_ dovecot-proxy/imap-login director root 16203 0.0 0.1 14604 2544 ? S 09:11 0:00 \_ dovecot-proxy/config webmail 16208 0.0 0.0 12804 1028 ? S 09:11 0:00 \_ dovecot-proxy/ipc root 7108 0.0 0.3 48240 6412 ? Ss Jul27 0:01 perl /usr/local/dovecot/bin/poolmon -- Regards, Alexandr Sabitov Systems Administrator Netregistry From mailinglists35 at gmail.com Mon Jul 30 13:36:49 2012 From: mailinglists35 at gmail.com (mailing list subscriber) Date: Mon, 30 Jul 2012 13:36:49 +0300 Subject: [Dovecot] maildir_copy_with_hardlinks on v.2.0.19 In-Reply-To: <58254FC4-5787-4FD1-8C15-36E20D860E44@iki.fi> References: <58254FC4-5787-4FD1-8C15-36E20D860E44@iki.fi> Message-ID: On Sat, Jul 28, 2012 at 8:04 PM, Timo Sirainen wrote: > On 23.7.2012, at 22.12, mailing list subscriber wrote: > >> As requested, here is my update. As you can see I am running now the >> latest release however emails delivered through lmtp gets split into >> different files instead of expected hardlinked files. > .. >> userdb { >> driver = passwd >> } > > Looks like you're using system users. Each mail then needs to be written using different permissions, so hard linking can't work. > I am afraid this is incorrect: [root at email ~]# cd /tmp [root at email tmp]# touch 1 [root at email tmp]# stat 1 File: `1' Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 803h/2051d Inode: 46923784 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-07-30 13:34:45.000000000 +0300 Modify: 2012-07-30 13:34:45.000000000 +0300 Change: 2012-07-30 13:34:45.000000000 +0300 [root at email tmp]# ln 1 2 [root at email tmp]# stat 2 File: `2' Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 803h/2051d Inode: 46923784 Links: 2 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2012-07-30 13:34:45.000000000 +0300 Modify: 2012-07-30 13:34:45.000000000 +0300 Change: 2012-07-30 13:34:51.000000000 +0300 [root at email tmp]# chown xfs:xfs 1 [root at email tmp]# stat 1 File: `1' Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 803h/2051d Inode: 46923784 Links: 2 Access: (0644/-rw-r--r--) Uid: ( 43/ xfs) Gid: ( 43/ xfs) Access: 2012-07-30 13:34:45.000000000 +0300 Modify: 2012-07-30 13:34:45.000000000 +0300 Change: 2012-07-30 13:35:03.000000000 +0300 [root at email tmp]# chown ntp:ntp 2 [root at email tmp]# stat 2 File: `2' Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 803h/2051d Inode: 46923784 Links: 2 Access: (0644/-rw-r--r--) Uid: ( 38/ ntp) Gid: ( 38/ ntp) Access: 2012-07-30 13:34:45.000000000 +0300 Modify: 2012-07-30 13:34:45.000000000 +0300 Change: 2012-07-30 13:35:15.000000000 +0300 [root at email tmp]# echo test > 2 [root at email tmp]# cat 1 test [root at email tmp]# From asier.cidon at externo.uam.es Mon Jul 30 10:06:26 2012 From: asier.cidon at externo.uam.es (asier.cidon at externo.uam.es) Date: Mon, 30 Jul 2012 09:06:26 +0200 Subject: [Dovecot] sieve vacation messages trouble Message-ID: <20120730090626.Horde.HxqjJmAflu1QFjJyA8Kia0A@webmail1.ti.uam.es> FROM:mailman-bounces at dovecot.org DATE:27. hein?kuuta 2012 16.58.35 UTC+3.00 TO:dovecot-owner at dovecot.org SUBJECT:CONTENT FILTERED MESSAGE NOTIFICATION The attached message matched the dovecot mailing list's content filtering rules and was prevented from being forwarded on to the list membership. ?You are receiving the only remaining copy of the discarded message. FROM:Asier Cidon Peria?ez DATE:27. hein?kuuta 2012 16.01.59 UTC+3.00 TO:dovecot at dovecot.org SUBJECT:RE: [DOVECOT] SIEVE VACATION MESSAGES TROUBLE Hi, I have a problem when I define a? Filter SIEVE vacation with Horde, Qmail and Dovecot. When I sent the message vacation for domains external, the log of postfix show <"<>"@mymachine>. I am writing you because I saw the post http://www.dovecot.org/list/dovecot/2009-June/040811.html Can you explain me whqt is the solution? Many Thanks. From punit_j at rediffmail.com Mon Jul 30 17:36:21 2012 From: punit_j at rediffmail.com (punit_j) Date: 30 Jul 2012 14:36:21 -0000 Subject: [Dovecot] =?utf-8?q?Enable_ACL_for_master_user?= Message-ID: <1343540280.S.4581.29256.H.TnB1bml0X2oAW0RvdmVjb3RdIEVuYWJsZSBBQ0wgZm9yIG1hc3RlciB1c2Vy.RU.rfs223, rfs223, 547, 218.f5-224-168.old.1343658981.10875@webmail.rediffmail.com> Hi , No one has a solution to this ? Regards, Punit From eray.aslan at caf.com.tr Mon Jul 30 17:56:35 2012 From: eray.aslan at caf.com.tr (Eray Aslan) Date: Mon, 30 Jul 2012 17:56:35 +0300 Subject: [Dovecot] managesieve segfault with gold linker In-Reply-To: <500024B8.2020500@caf.com.tr> References: <4FFF3326.9060802@caf.com.tr> <4FFFF01A.9020402@rename-it.nl> <500024B8.2020500@caf.com.tr> Message-ID: <5016A0A3.1020201@caf.com.tr> On 07/13/2012 04:38 PM, Eray Aslan wrote: > I'll try to find out more but I will be on vacation for a few days > so it might not be quick. Recap: When linked with gold, pigeonhole-0.3.1 make check fails with: dovecot-2.1-pigeonhole-0.3.1/src/testsuite/.libs/lt-testsuite: symbol lookup error: dovecot-2.1-pigeonhole-0.3.1/src/lib-sieve/.libs/libdovecot-sieve.so.0: undefined symbol: mail_deliver_get_log_var_expand_table Well, /usr/lib64/dovecot/libdovecot-lda.so provides the mail_deliver_get_log_var_expand_table symbol, but /usr/lib64/dovecot/libdovecot-sieve.so doesn't link against it. The following patch works for me: --- src/lib-sieve/Makefile.am 2012-05-11 03:38:47.000000000 +0300 +++ src/lib-sieve/Makefile.am 2012-07-30 17:38:15.878643313 +0300 @@ -70,8 +70,8 @@ $(extdir)/vnd.dovecot/duplicate/libsieve_ext_duplicate.la \ $(unfinished_plugins) -libdovecot_sieve_la_DEPENDENCIES = $(plugins) -libdovecot_sieve_la_LIBADD = $(plugins) +libdovecot_sieve_la_DEPENDENCIES = $(plugins) $(LIBDOVECOT_LDA_DEPS) +libdovecot_sieve_la_LIBADD = $(plugins) $(LIBDOVECOT_LDA) libdovecot_sieve_la_SOURCES = \ rfc2822.c \ -- Eray Aslan From jtam.home at gmail.com Tue Jul 31 02:08:21 2012 From: jtam.home at gmail.com (Joseph Tam) Date: Mon, 30 Jul 2012 16:08:21 -0700 (PDT) Subject: [Dovecot] expire-tool stops on first box with nothing to expire in it Message-ID: Timo Sirainen writes: > > In practice, if this query is done on a periodic interval (e.g. daily), > > the date.saved will be roughly correct, and is usable as an age test > > for expungement. > > > > Is there a way to force this value to be cached when the client actually > > saves or moves a message? > > Done for save: http://hg.dovecot.org/dovecot-2.1/rev/c598f76eeeed > > Fixed copy: http://hg.dovecot.org/dovecot-2.1/rev/b13b8267999d > > It still doesn't add it to cache if it's not already cached in both the > source and destination mailbox, but usually it should be after initial > doveadm run. Thanks, Timo. Joseph Tam From matthew at atom.net Tue Jul 31 03:07:39 2012 From: matthew at atom.net (Matthew Powell) Date: Mon, 30 Jul 2012 20:07:39 -0400 Subject: [Dovecot] Address extensions In-Reply-To: References: Message-ID: <9CA68447-5D12-46AC-B6DC-7B819766F432@atom.net> On 2012-07-29, at 3.01, "Armistead, Kurt" wrote: > I have re-defined the delivery service in master.cf as "-f ${sender} -d > ${user}@${nexthop} -n -m INBOX.${extension}" (server was formerly courier) > but the routing only works when the destination folder has no uppercase > characters. "Folder1" doesn't deliver correctly while "folder2" does. Have a look at http://wiki.dovecot.org/LDA/Postfix. It sounds like you have the 'u' flag set, which squashes username+extension to lower case. -- Matthew Powell matthew at atom.net From matthias.lay at securepoint.de Tue Jul 31 12:42:20 2012 From: matthias.lay at securepoint.de (Matthias Lay) Date: Tue, 31 Jul 2012 11:42:20 +0200 Subject: [Dovecot] authenticate plain and utf-8 with special chars Message-ID: <1343727740.3072.28.camel@hermann.potsdam.securepoint.local> Hi, I experienced some problems with authenticate. seems like the mail clients like thunderbird send their base64 string in ISO- encoding, which doesnt work on my dovecot setup if there are some Umlauts I am not able to login as "?mer" for example If I authenticate from telnet with an base64 encoded utf-8 string like 'echo -en "\0?mer\0Start12" |base64' everything works fine -dovecot 2.1.7 -userdb is an openldap which queries an windows AD so results are in utf8, I think. any clues at which point to fix this are welcome ;) From gvgter at googlemail.com Tue Jul 31 18:35:42 2012 From: gvgter at googlemail.com (GVG GVG) Date: Tue, 31 Jul 2012 17:35:42 +0200 Subject: [Dovecot] openbsd + dovecot not downloading emails Message-ID: Dear list members, first my doveconf -n output: -------------- # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: OpenBSD 5.1 i386 ffs mail_location = mbox:/var/mail:INBOX=/var/mail/%u mbox_write_locks = fcntl mmap_disable = yes passdb { driver = bsdauth } ssl_cert = References: Message-ID: <50181B7C.3010606@localhost.localdomain.org> On 07/31/2012 05:35 PM GVG GVG wrote: > Dear list members, > > first my doveconf -n output: > > -------------- > # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf > # OS: OpenBSD 5.1 i386 ffs > mail_location = mbox:/var/mail:INBOX=/var/mail/%u > mbox_write_locks = fcntl > mmap_disable = yes > passdb { > driver = bsdauth > } > ssl_cert = ssl_key = userdb { > driver = passwd > } > protocol imap { > imap_client_workarounds = delay-newmail tb-extra-mailbox-sep tb-lsub-flags > } > protocol pop3 { > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > } > --------------- > > when using evolution from a remote linux machine, and try to download/fetch > the e-mails from my e-mail server, I get a pop-up window indicationg that > is trying to do so (after I did sign-in with my user-id and password!). The > problem is taht this process stays there forever and no messages get > downloaded! Your mail_location setting looks a little bit strange to me. I'd suggest to use: mail_location = mbox:~/mail:INBOX=/var/mail/%u If that shouldn't solve the issue, make Dovecot more verbose: mail_debug = yes Regards, Pascal -- The trapper recommends today: c01dcafe.1221301 at localdomain.org From slitt at troubleshooters.com Tue Jul 31 21:29:21 2012 From: slitt at troubleshooters.com (Steve Litt) Date: Tue, 31 Jul 2012 14:29:21 -0400 Subject: [Dovecot] [GoLugTech] thunderbird In-Reply-To: References: Message-ID: <20120731142921.02be1a72@mydesk> On Mon, 30 Jul 2012 23:43:49 -0400, Chris Clifton said: > Hi all, > > Do I read it correctly that thunderbird funding is being cut back by > mozilla? > > I'm not on a linux mail client yet, wondering what others are using in > ubuntu for mail client, I need multiple pop3 profiles, good html > support, fast indexing and searching, solid gpg support and all the > usual bells and whistles. Evolution seems to be in pretty widespread > use. Wondering what others like and use heavily. Hi Chris, I tried Evolution once about 10 years ago, and it was too complex and comprehensive for me to understand. The centerpiece of my emailing activity is a Dovecot IMAP server on my daily driver desktop computer. The way it works is that fetchmail grabs all my email from my various POP accounts, pushes them through procmail, whose filters distribute the emails into the proper Dovecot IMAP directories (and also /dev/null stuff from PITA people). As far as I know, my Dovecot has never malfunctioned. Currently, I access my local IMAP with Claws-Mail. Claws is capable, robust, and does what I need without being an ecological disaster like Kmail or intermittently pig slow like Thunderbird. Claws' configuration is difficult to wrap your head around, and very much in need of better documentation, but so far, thanks to a knowledgeable and helpful user community on the mailing list, I've been able to make almost every config change to speed my interaction with Claws. My one complaint about Claws that can't be cured by documentation is its propensity to do everything with one process and one thread, so while you're sending an email or scanning your IMAP, you cannot look at other emails. I also wanted their powerful search to be able to recurse folders, and when nobody volunteered to do it, I tried to do it myself. Making something recursive sounds like simply a matter of writing a simple loop around the existing functionality, but what I found out is, at least this part of Claws was written in a manner resembling MVC, with process and UI completely commingled. For that reason I was unable to make it recursive. I have Claws on my daily driver and all my laptops. Thanks to Kevin Korb's instructions, on the laptops I was able to tell Claws its IMAP server was at 127.0.0.1. I pinholed my OpenBSD/pf firewall appliance to port forward incoming ssh to my daily driver. Then, on my laptop, if I'm at home and on the LAN, I run the following inhouse.sh sudo ssh -NTL 993:127.0.0.1:993 slitt at 192.168.1.88 While on the road I run this travelling.sh: sudo ssh -NTL 993:127.0.0.1:993 slitt at 99.99.99.99 In the preceding, 99.99.99.99 is the IP address my ISP gave me, while 192.168.1.88 is the LAN IP address of my daily driver. This makes 127.0.0.1:993 get pushed through ssh to the ISP supplied internet address or the LAN address. So I can use Claws to interact with my daily driver desktop anywhere I go. I'm extremely pleased with this setup. My IMAP tree is trivial to back up, and gives me the assurance that if Claws-Mail ever commits suicide the way Kmail did when they went to Kmail2, I can simply plug in a different IMAP enabled email client. Meanwhile, Claws gives me an extremely productive and easy to use user interface. With my client/server setup, I no longer need to copy all my email too and from the laptop when travelling. I'm going to present on my remote use of my daily driver desktop Dovecot, using Claws and ssh, at tomorrow night's GoLUG meeting. HTH, SteveT Steve Litt * http://www.troubleshooters.com/ * http://twitter.com/stevelitt Troubleshooting Training * Human Performance From gvgter at googlemail.com Tue Jul 31 21:31:02 2012 From: gvgter at googlemail.com (GVG GVG) Date: Tue, 31 Jul 2012 20:31:02 +0200 Subject: [Dovecot] openbsd + dovecot not downloading emails In-Reply-To: <50181B7C.3010606@localhost.localdomain.org> References: <50181B7C.3010606@localhost.localdomain.org> Message-ID: here is my new doveconf-n: ------------------ # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf # OS: OpenBSD 5.1 i386 auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = sha1 mail_debug = yes mail_location = mbox:~/mail:INBOX=/var/mail/%u mbox_write_locks = fcntl mmap_disable = yes passdb { driver = bsdauth } ssl_cert = , method=PLAIN, rip=x.x.x.x, lip=x.x.x.x, mpid=26976, TLS --------------------------- Thanks On Tue, Jul 31, 2012 at 7:53 PM, Pascal Volk < user+dovecot at localhost.localdomain.org> wrote: > On 07/31/2012 05:35 PM GVG GVG wrote: > > Dear list members, > > > > first my doveconf -n output: > > > > -------------- > > # 2.0.17 (684381041dc4+): /etc/dovecot/dovecot.conf > > # OS: OpenBSD 5.1 i386 ffs > > mail_location = mbox:/var/mail:INBOX=/var/mail/%u > > mbox_write_locks = fcntl > > mmap_disable = yes > > passdb { > > driver = bsdauth > > } > > ssl_cert = > ssl_key = > userdb { > > driver = passwd > > } > > protocol imap { > > imap_client_workarounds = delay-newmail tb-extra-mailbox-sep > tb-lsub-flags > > } > > protocol pop3 { > > pop3_client_workarounds = outlook-no-nuls oe-ns-eoh > > } > > --------------- > > > > when using evolution from a remote linux machine, and try to > download/fetch > > the e-mails from my e-mail server, I get a pop-up window indicationg that > > is trying to do so (after I did sign-in with my user-id and password!). > The > > problem is taht this process stays there forever and no messages get > > downloaded! > > Your mail_location setting looks a little bit strange to me. > I'd suggest to use: mail_location = mbox:~/mail:INBOX=/var/mail/%u > > If that shouldn't solve the issue, make Dovecot more verbose: > mail_debug = yes > > > Regards, > Pascal > -- > The trapper recommends today: c01dcafe.1221301 at localdomain.org > From tss at iki.fi Tue Jul 31 21:34:37 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 21:34:37 +0300 Subject: [Dovecot] [GoLugTech] thunderbird In-Reply-To: <20120731142921.02be1a72@mydesk> References: <20120731142921.02be1a72@mydesk> Message-ID: <1343759677.18782.15.camel@innu> On Tue, 2012-07-31 at 14:29 -0400, Steve Litt wrote: > My one complaint about Claws that can't be cured by documentation is > its propensity to do everything with one process and one thread, so > while you're sending an email or scanning your IMAP, you cannot look at > other emails. I also wanted their powerful search to be able to recurse > folders, and when nobody volunteered to do it, I tried to do it myself. You could also do this on Dovecot side by creating a virtual folder out of your all folders and do the search there: http://wiki2.dovecot.org/Plugins/Virtual From tss at iki.fi Tue Jul 31 21:42:55 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 21:42:55 +0300 Subject: [Dovecot] maildir_copy_with_hardlinks on v.2.0.19 In-Reply-To: References: <58254FC4-5787-4FD1-8C15-36E20D860E44@iki.fi> Message-ID: <9AAFF879-213B-48F7-8B1A-9A900338919A@iki.fi> On 30.7.2012, at 13.36, mailing list subscriber wrote: >>> userdb { >>> driver = passwd >>> } >> >> Looks like you're using system users. Each mail then needs to be written using different permissions, so hard linking can't work. >> > > I am afraid this is incorrect: The code that determines if Dovecot even attempts to use hard links is in: http://hg.dovecot.org/dovecot-2.1/file/f74557336910/src/lib-storage/mail-copy.c#l101 So you could make it work if you chmod 0770 all of your maildir directories and chgrp them to the same group. From tss at iki.fi Tue Jul 31 21:45:35 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 21:45:35 +0300 Subject: [Dovecot] Problem with running two dovecot instances on the same server In-Reply-To: <501636C0.5070305@netregistry.com.au> References: <5010AB05.8000709@netregistry.com.au> <1F7239B6-9E6D-4CB8-8B51-BB9CDF000199@iki.fi> <501636C0.5070305@netregistry.com.au> Message-ID: On 30.7.2012, at 10.24, Alexandr Sabitov wrote: >>> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Error: Empty server list >>> Jul 26 11:51:20 dovecot-test-1 dovecot: director: Fatal: Invalid value for director_mail_servers setting >>> Jul 26 11:51:20 dovecot-test-1 dovecot: master: Error: service(director): command startup failed, throttling for 60 secs >> Director process gets started for the backend installation where it of course isn't configured. My guess: doveadm (from some cronjob?) is attempting to connect to /var/run/dovecot/director-admin instead of /var/run/dovecot-proxy/director-admin. > > Hi Timo, > > Thanks for reply. > > That's right: strace shows that director process is a child of non-proxy dovecot (pid 7089) which is should not be happening. And I don't have any cronjob running. > > Might we need just update dovecot wiki that if someone runs dovecot and dovecot-proxy then they have to be installed in different folders as > separate binaries otherwise one instance cross over another somehow magically. :) I have several such setups and they don't have this problem, so you've managed to create something that tries to connect there. One easy way to make sure this never happens is to just disable the socket: service director { unix_listener director-admin { mode = 0 } } From tss at iki.fi Tue Jul 31 21:47:27 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 21:47:27 +0300 Subject: [Dovecot] Help needed with configuration giving dict error No such file In-Reply-To: References: Message-ID: <16CA0919-FC93-49ED-8B82-D2D6A34FC40F@iki.fi> On 29.7.2012, at 22.09, Ajax wrote: > file_dotlock_open(/var/vmail/XXX.com/jax/home/maildir/dovecot-quota) > failed: No such file or directory The /var/vmail/XXX.com/jax/home/maildir/ directory doesn't exist. > quota = dict:user::file:%h/maildir/dovecot-quota This should be %h/dovecot-quota. I'm not entirely sure if Dovecot autocreates the home directory. From tss at iki.fi Tue Jul 31 21:51:24 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 21:51:24 +0300 Subject: [Dovecot] BUG: LDAP extension failed to authenticate if 'base' DN configuration is empty '' In-Reply-To: <34226738.post@talk.nabble.com> References: <34226738.post@talk.nabble.com> Message-ID: On 29.7.2012, at 15.00, Hendy Irawan wrote: > We have several context entries and want to authenticate a user against the > whole LDAP server. > > However when setting "base = " Dovecot LDAP fails with following message: > > Jul 29 10:50:59 nitik1 dovecot: auth: Error: > ldap(hendy at staging.member.berbatik.com,127.0.0.1): > ldap_search((virtualMail=hendy at staging.member.berbatik.com)) failed: Other > (e.g., implementation specific) error Does the attached patch help? If yes, I'll add it to next version. -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 533 bytes Desc: not available URL: From tss at iki.fi Tue Jul 31 21:54:33 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 21:54:33 +0300 Subject: [Dovecot] Dovecot Director behaviour when two or more fail In-Reply-To: <5014BC74.10006@optusnet.com.au> References: <5014BC74.10006@optusnet.com.au> Message-ID: <5FEC6F3A-E355-428C-9037-A8D3B1C05071@iki.fi> On 29.7.2012, at 7.30, Dominic Malolepszy wrote: > On the Dovecot wiki here http://wiki2.dovecot.org/Director it says the following: > > "The directors are going to connect to each others in a ring. For example if you have servers called A, B and C, director will create connections A->B, B->C and C->A" > > I am curious what happens in a scenario when two or more directors fail and potentially break a ring in half, do the active directors detect this and reform a new ring? Directors always attempt to create the ring as fully as they can, doesn't matter how many servers die. From tss at iki.fi Tue Jul 31 22:04:02 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 22:04:02 +0300 Subject: [Dovecot] fts solr : out of memory In-Reply-To: <501473C0.2080106@starbridge.org> References: <50140387.70805@starbridge.org> <43E45374-4BD1-4F45-B389-3EE3D6236A33@iki.fi> <501473C0.2080106@starbridge.org> Message-ID: On 29.7.2012, at 2.20, tonio at starbridge.org wrote: > it seems ok now with memory problem, but an other error shows up: > > doveadm -v index -u clxx at spamguard.fr INBOX > doveadm(clxx at spamguard.fr): Info: INBOX: Caching mails seq=1..59363 > 21500/59363doveadm(clxx at spamguard.fr): Error: fts_solr: Invalid XML > input at line 1: mismatched tag No idea. You can reproduce this? What does it log with this patch? http://hg.dovecot.org/dovecot-2.1/rev/817b69b2b21f > doveadm(clxx at spamguard.fr): Panic: file solr-connection.c: line 545 > (solr_connection_post_more): assertion failed: (maxfd >= 0) I guess related to the previous error, but I'm not really sure why it happens either. From tss at iki.fi Tue Jul 31 22:10:22 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 22:10:22 +0300 Subject: [Dovecot] ntlm auth / usernames with white spaces In-Reply-To: <1343399579.3072.19.camel@hermann.potsdam.securepoint.local> References: <1343399579.3072.19.camel@hermann.potsdam.securepoint.local> Message-ID: On 27.7.2012, at 17.32, Matthias Lay wrote: > we experienced some problems with white spaces in usernames (this was > NOT our idea) authenticated bei dovecots ntlmhelper > > I wrote a little patch fixing this, if you would like. > > patch applies on 2.1.7 and 2.1.8 (tested those) Does this simpler patch also work? -------------- next part -------------- A non-text attachment was scrubbed... Name: diff Type: application/octet-stream Size: 430 bytes Desc: not available URL: From tss at iki.fi Tue Jul 31 22:30:42 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 22:30:42 +0300 Subject: [Dovecot] Empty argument in "executable = script ..." service In-Reply-To: <20120726132656.4f909f6e@auedv3.syscomp.de> References: <20120726132656.4f909f6e@auedv3.syscomp.de> Message-ID: <5C5EDC89-129A-4854-887D-04C2F217680A@iki.fi> On 26.7.2012, at 14.26, Florian Wagner wrote: > what is the point of adding an empty argument to the command called by > the "executable = script ..." setting in a service? I guess that would > be the call to array_append_space at line 57 in util/script.c. Fixed: http://hg.dovecot.org/dovecot-2.1/rev/1d9d799a2efc From tss at iki.fi Tue Jul 31 22:33:47 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 22:33:47 +0300 Subject: [Dovecot] improve error messages with quotes In-Reply-To: <50082FA2.5050106@filez.com> References: <50082FA2.5050106@filez.com> Message-ID: <4C23DB7C-2D01-493C-8540-7FD7BCD35972@iki.fi> On 19.7.2012, at 19.02, Radim Kolar wrote: > I propose to improve error messages but inserting quotes. I spent good time until I looked into source code to find right meaning of error message. > > I would like them in following format: > > namespace configuration error no 'hidden=no' namespaces I think this is clearer: http://hg.dovecot.org/dovecot-2.1/rev/0abb9b0a54db From tss at iki.fi Tue Jul 31 22:39:36 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 22:39:36 +0300 Subject: [Dovecot] mdbox with separate index location results in many emptydirectories In-Reply-To: <20120720144604.12933ye5jvozyeyo@webmail.hrz.uni-giessen.de> References: <20120720144604.12933ye5jvozyeyo@webmail.hrz.uni-giessen.de> Message-ID: <96465E5A-164D-4CE6-9A89-F48BE344EA9F@iki.fi> On 20.7.2012, at 15.46, J?rgen Obermann wrote: > mail_location = mdbox:/dovecot/storage%h:INDEX=/dovecot/index%h .. > As expected the index files now reside under /dovecot/index%h/mailboxes/, but the path /dovecot/storage%h/mailboxes/ also contains for every user and every mailbox name a directory which in turn contains only an empty subdirectory 'dbox-Mails'. These directories seem to me to sum up to a lot of unneded infrstructure objects. I tried to delete these empty directories, but dovecot did not like that and stopped to work. So somehow they are necessary, in spite of setting a separate index file location. > > So my question is, could these empty directories be avoided, perhaps in a future version? No plans to do it. The empty directories under /dovecot/storage are used to find out what mailboxes exist. And in some setups those directories aren't always empty (e.g. dovecot-acl files go there). From tss at iki.fi Tue Jul 31 22:42:58 2012 From: tss at iki.fi (Timo Sirainen) Date: Tue, 31 Jul 2012 22:42:58 +0300 Subject: [Dovecot] FETCH Error In-Reply-To: <5006B0D4.9010602@krausam.de> References: <5006B0D4.9010602@krausam.de> Message-ID: <45F954EB-F82C-4C19-B18B-0F6A1DDCB940@iki.fi> On 18.7.2012, at 15.49, Micha Krause wrote: > Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: FETCH [] for mailbox INBOX.Sent UID 166 got too little data: 120860 vs 120867 > Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: Maildir filename has wrong W value: /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003,W=284007:2,S > Jul 18 14:36:09 mailint1 dovecot: imap(micha): Error: Corrupted index cache file /var/mail/mailboxes/users/micha/.Sent/dovecot.index.cache: Broken virtual size for mail UID 166 All of those numbers seem wrong.. > so far i tried deleting: > > /var/mail/mailboxes/users/micha/.Sent/dovecot.index.cache > /var/mail/mailboxes/users/micha/.Sent/dovecot.index > /var/mail/mailboxes/users/micha/.Sent/dovecot.index.log > > I tried to renaming: > /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003,W=284007:2,S to > /var/mail/mailboxes/users/micha/.Sent/cur/1240390202.Rfd9752b8Q14135.mail-alt,S=280003:2,S How large is that file really? I think you need to both rename it and delete dovecot.index.cache file. If the S=value is also wrong you'd need to delete it too. From tonio at starbridge.org Tue Jul 31 22:46:54 2012 From: tonio at starbridge.org (tonio at starbridge.org) Date: Tue, 31 Jul 2012 21:46:54 +0200 Subject: [Dovecot] fts solr : out of memory In-Reply-To: References: <50140387.70805@starbridge.org> <43E45374-4BD1-4F45-B389-3EE3D6236A33@iki.fi> <501473C0.2080106@starbridge.org> Message-ID: <5018362E.30003@starbridge.org> Le 31/07/2012 21:04, Timo Sirainen a ?crit : > On 29.7.2012, at 2.20, tonio at starbridge.org wrote: > >> it seems ok now with memory problem, but an other error shows up: >> >> doveadm -v index -u clxx at spamguard.fr INBOX >> doveadm(clxx at spamguard.fr): Info: INBOX: Caching mails seq=1..59363 >> 21500/59363doveadm(clxx at spamguard.fr): Error: fts_solr: Invalid XML >> input at line 1: mismatched tag > No idea. You can reproduce this? What does it log with this patch? http://hg.dovecot.org/dovecot-2.1/rev/817b69b2b21f It happens every time on the same mailboxes (very few) around the same uid number (I think I can find the exact uid with strace and send the email message to you if it helps) catalina.out show this at this time: INFO: {} 0 1 31 juil. 2012 21:19:56 org.apache.solr.common.SolrException log GRAVE: org.apache.solr.common.SolrException: Illegal character ((CTRL-CHAR, code 4)) at [row,col {unknown-source}]: [533,500] at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:81) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:58) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1376) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:365) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Unknown Source) Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Illegal character ((CTRL-CHAR, code 4)) at [row,col {unknown-source}]: [533,500] at com.ctc.wstx.sr.StreamScanner.throwInvalidSpace(StreamScanner.java:675) at com.ctc.wstx.sr.BasicStreamReader.readTextSecondary(BasicStreamReader.java:4668) at com.ctc.wstx.sr.BasicStreamReader.readCoalescedText(BasicStreamReader.java:4126) at com.ctc.wstx.sr.BasicStreamReader.finishToken(BasicStreamReader.java:3701) at com.ctc.wstx.sr.BasicStreamReader.safeFinishToken(BasicStreamReader.java:3649) at com.ctc.wstx.sr.BasicStreamReader.getText(BasicStreamReader.java:809) at org.apache.solr.handler.XMLLoader.readDoc(XMLLoader.java:315) at org.apache.solr.handler.XMLLoader.processUpdate(XMLLoader.java:156) at org.apache.solr.handler.XMLLoader.load(XMLLoader.java:79) ... 17 more 31 juil. 2012 21:19:56 org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/update params={} status=400 QTime=1 After a quick google search , it seems related to invalid Control Character sent to SOLR. I've applied your last patch and the message is now: Error: fts_solr: Invalid XML input at 4:113: mismatched tag (near: Apache Tomcat/6.0.35 - Rapport d'erreur