- dovecot - dovecot.org

[Dovecot] dsync, hard-links and refcounts
by Christoph Bußenius 21 Feb '12

21 Feb '12

Hi, when creating a copy of a mail, dovecot provides a feature that will store its contents only once. In maildir, this is done by means of hard links, while mdbox has some special refcounting mechanism. My question is, how can we convert mails from maildir to mdbox without duplicating these copies? It seems that dsync does not detect the hard links. Even if the hard-linked mails have the same GUID listed in dovecot-uidlist, dsync creates multiple instances of the text. - Is there some way to make dsync notice the hard links? (I used dovecot 2.0.17) - Alternatively, is there some tool to deduplicate the mdbox after it has been converted from maildir? - NB: I am not talking about single-instance storage for attachments. Thank you very much, Christoph -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

2 10

[Dovecot] Managesieve AUTHENTICATE with IDN using UTF-8 or encode into ACE?
by Christian Rohmann 21 Feb '12

21 Feb '12

Hey dovecot-users, clients that speak telnet based protocols like http, smtp usually encode IDN (International Domain Names) containing UTF-8 characters into ACE (ASCII Compatible Encoding) punycode. I am wondering what the case was with managesieve (RFC 5804). For example if I was using a username like "user(a)xn--4ca0br.example.com" with the AUTHENTICATE command. Would that need to be encoded into ACE or be left in UTF-8? Reading section "2.1. AUTHENTICATE Command" or "4. Formal Syntax" suggests that quoted or literal UTF-8 is correct. Thanks Christian

2 1

[Dovecot] Some trouble with dovecot 2.1 and SASL
by Alessio Cecchi 21 Feb '12

21 Feb '12

Hi, I'm testing dovecot 2.1.0, on Debian 6.0, with "passdb driver = imap" and SASL for postfix's SMTP-Auth, like explained ih this howto: http://wiki2.dovecot.org/HowTo/PostfixAndDovecotSASL Authentications over IMAP (local dovecot -> remote dovecot) works fine, and it's great. Also smtp-auth on postfix (over SASL) works fine. But in the dovecot log I found this error: Feb 19 11:11:42 demo-vpop dovecot: auth: Fatal: master: service(auth): child 30506 killed with signal 11 (core dumps disabled) even though all works fine. This is my dovecot configuration: # 2.1.0: dovecot.conf-imapc # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.4 auth_mechanisms = plain login default_login_user = nobody imapc_host = 10.0.0.1 mail_gid = imapproxy mail_home = /home/imapproxy/%u mail_location = imapc:~/imapc mail_uid = imapproxy passdb { args = host=10.0.0.1 default_fields = userdb_imapc_user=%u userdb_imapc_password=%w driver = imap } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } ssl = no userdb { driver = prefetch } But if instead of "unix_listener /var/spool/postfix/private/auth" I use: service auth { unix_listener auth-userdb { } inet_listener { port = 12345 } } no errors appears in the log. In both cases all works fine. Is there any errors in my dovecot's configuration? -- Alessio Cecchi is: on LinkedIn -> http://www.linkedin.com/in/alessice Assistenza Sistemi Linux -> http://www.cecchi.biz/ Hosting Linux -> http://www.hosting-linux.it @ PLUG -> ex-Presidente, adesso senatore a vita, http://www.prato.linux.it

3 4

Re: [Dovecot] Questions still about a uw-imap -> dovecot conversion
by Steve Campbell 21 Feb '12

21 Feb '12

On 2/20/2012 4:58 PM, Jim Lawson wrote: > On 2/20/12 3:36 PM, Steve Campbell wrote: >> >> Thanks for that input. I still think I'm missing something since I >> too used the compatibility link that you pointed to. Only thing is >> that proceeding those namespaces, I used the first example of: >> >> namespace { >> type = private >> separator = / >> prefix = "#mbox/" >> location = mbox:~/mail:INBOX=/var/mail/%u >> inbox = yes >> hidden = yes >> list = no >> } >> namespace { >> type = private >> separator = / >> prefix = >> location = maildir:~/mail >> } >> >> This seemed to get the ball rolling so that users could access >> anything at all. I still have the problem of client imap folders >> being different from webmail imap folders. That's pretty much why I'm >> thinking of using mbox as the INBOX and all other imap folders in the >> ~/mail directory in maildir format. > You ought to be able to get webmail to see the same folders as the > imap clients. > >> >> Should I remove the first two namespaces, in your opinion? Right now, >> the whole thing is kind of fragile. >> > > From your added namespaces, it looks like you are trying to support > mbox and maildir and ~/mail at the same time. Do you already have > maildir folders to support? If not, I would try to get things working > well with mbox first before I started a conversion to maildir. I > would also ask why you're thinking of moving to maildir. Yes, there > are caching benefits, but when you add the Dovecot indexing on top of > mbox, it's pretty much a wash. If you are using file-level backup, > rather than some sort of snapshot technology, maildir will be much, > much slower to back up. Your system will spend all its time walking > directories, opening and closing files. If you don't have many users > to worry about it might be OK, but make sure it's worth it. A lot of > sites went to maildir in the 1998-2004 era and have regretted the > decision as their systems get overloaded with files and they can't > back them up. No, I've got all mbox formats. Previous posts probably suggested that I wanted to move to maildir, but all of the replies I've received have convinced me that I do not want to do that. The folders in ~ and ~/mail are mboxes, so I need to see what damage I've done with a maildir namespace. The maildir reference could be part of the problems I'm seeing. I'm still not sure whether I should be seeing .subscriptions or mail/subscriptions anywhere and whether Dovecot will use the .mailboxlist that exists. The wiki suggests that I need to recompile Dovecot to continue using .mailboxlist. This is something I don't want to do. > >> Horde/Imp updates are probably out of the question until I can get a >> server to install the upgrade on. > For Dovecot and IMP both, you should set up an alternate server to > test out your config changes on before you put them into production. > If you are running on a bare metal single server, set up another > Dovecot instance on a different set of ports (I commonly use 20143 > (imap), 20993 (imaps), 20110 (pop3)) which you can fiddle with > freely. Once you are satisfied with the result in your various > clients, put it into production. You can do the same with Horde/IMP > by putting an installation in a different location on your webserver. > > Jim I've got a second server that is totally independent of the one I'm messing with. I've done the horde/imp alternate port/config. It all went well. The downside is that I didn't realize those secondary folders in ~/mail were being hidden. Kinda late to switch back, but the secondary server supports another domain, so I have it to test with. Again, the damage I've caused to those secondary imap folders needs to be determined to get this fixed properly. Most users are seing the secondary folders and can use them. It's just those users who have folders in ~ that are seeing problems as far as I can tell. Thanks steve > >

2 1

Re: [Dovecot] Questions still about a uw-imap -> dovecot conversion
by Joseph Tam 21 Feb '12

21 Feb '12

Steve Campbell wrote: > The more I read about all of this, the more I'm thinking about moving to > maildir format. My switchover this weekend is full of holes due to the > way user's imap folders were laid out. Some had folders in their home > directory and others might have folders in their /home/mail directory. I had the same problem, and handled it more or less the same way as the later poster (Jim Lawson <jtl+dovecot(a)uvm.edu>) handled his site. I retained use of mbox format as well. I planned the migration like this - scan user home directories for mailboxes (especially "Trash", "Drafts", "Sent", etc.) looking for "From " as the first 5 bytes. This was piped into a script that Email'd users about the changes that was going to happen, what they would expect to see, and a FAQ on how to set up a mail client correctly. - during the cutover, - mailboxes left on the home directory were moved to ~/mail, or renamed (e.g. "Sent" -> "00Sent" to avoid name collision for users that had a mixture of correct and blank prefixes. (I think I deleted "Trash", "Junk", etc. anyways). - Namespace aliasing was used so that prefixes "", "mail", "~/mail", and whatever darn fool settings my users used, would map to the same directory. - .subscription files were moved into the mail folder (don't have to edit prefixes since the aliasing will take care of that). - depending on what POP3 client you used (I used qpopper), you may need to configure pop3_reuse_xuidl = yes to avoid a massive re-downloading from POP3 clients after cutover. - the mail clients I control centrally (e.g. webmail, public server mail clients, etc). shouldn't need updating since they ought to have been set up properly in the first place. - after cutover, a second notification was sent for users that didn't move their mailboxes the first time around, and was it done for them during migration. And of course, test like crazy and watch the logs like a hawk. Other gotcha's: - your setup is fairly close to mine, so you may also run into the problem of user having mailboxes with group ownership that users are not part of (for example, group "mail" for INBOX set by your LDA or personal mailboxes with groups the user is no longer a member of) *and* with group permission not mode=0. You'll have to treat these (set mode=0, or change the group to something the user is part of), or the dovecot index creation will fail and they won't be able to access their mailboxes. I think my migration went pretty smoothly. Less than a handful of wazzup' Email problem reports. Joseph Tam <jtam.home(a)gmail.com>

2 1

[Dovecot] Recalculate quota when quota=dict ?
by Jacek Osiecki 21 Feb '12

21 Feb '12

Hi, My current setting for quota is: plugin { #quota = dirsize:User quota #quota = maildir:User quota quota = dict:User quota::proxy::quota #quota = fs:User quota } I have used mb2md.pl and copied result files manually to a desired mail folder - but quota was not recalculated. And it seems, that no matter what I do - the quota usage remains the same. I tried uncommenting line with: quota = maildir:User quota but it did not help. Any other idea how to recalculate quota? Greetings, -- Jacek Osiecki joshua(a)ceti.pl GG:3828944 I don't want something I need. I want something I want.

4 6

[Dovecot] passwd-file - lower limit for username?
by Oon-Ee Ng 21 Feb '12

21 Feb '12

Hi, previously I could use the following in my passwd-file:- AB:{plain}password I'm not sure when it stopped working, its not an account I access with any regularity, but today it would not work (2.1.0, 64-bit Arch Linux), with an 'unknown user' error in the logs. Is there some new lower limit for username length?

2 3

Re: [Dovecot] dovecot freezes when trying to get mail from maildir with mail
by postmaster 21 Feb '12

21 Feb '12

From: Timo Sirainen <tss(a)iki.fi> > #9 0x0007ba2f in mail_index_update_day_headers () So Dovecot calls localtime() and the process hangs. Something's badly broken in your system related to timezone handling, nothing Dovecot can do about it.. OK - Thanks for your help :-) I'll take it up on the macports mailing lists then ! -- Bjarne D Mathiesen København N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit miljø MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3

1 0

Re: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail
by postmaster 21 Feb '12

21 Feb '12

[ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen København N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit miljø MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 arrrrgh -- used pop3 instead of imap :-( 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/imap -u postmaster(a)mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/imap -u postmaster(a)mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster(a)mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/, alt= * PREAUTH [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in as postmaster(a)mathiesen.info a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x00092a2f in mail_index_update_day_headers () No symbol table info available. #10 0x0008ec95 in mail_index_transaction_finish () No symbol table info available. #11 0x0008d80a in mail_index_transaction_commit_v () No symbol table info available. #12 0x0008dacf in mail_index_transaction_commit_full () No symbol table info available. #13 0x0008db18 in mail_index_transaction_commit () No symbol table info available. #14 0x00096434 in mail_index_sync_commit () No symbol table info available. #15 0x00066f02 in maildir_sync_index_finish () No symbol table info available. #16 0x00068376 in maildir_sync_context () No symbol table info available. #17 0x000686f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000295fd in mailbox_sync () No symbol table info available. #19 0x00007833 in cmd_select_full () No symbol table info available. #20 0x00007d49 in cmd_select () No symbol table info available. #21 0x00009f63 in client_command_input () No symbol table info available. #22 0x0000a0c2 in client_command_input () No symbol table info available. #23 0x0000a1d3 in client_handle_input () No symbol table info available. #24 0x000095bd in client_input () No symbol table info available. #25 0x001171a7 in io_loop_call_io () No symbol table info available. #26 0x00118492 in io_loop_handler_run () No symbol table info available. #27 0x00117148 in io_loop_run () No symbol table info available. #28 0x0010425a in master_service_run () No symbol table info available. #29 0x00013c5a in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root#

1 0

Re: [Dovecot] [dovecot] dovecot freezes when trying to get mail from maildir with mail
by postmaster 21 Feb '12

21 Feb '12

[ I'm sorry I can't reply in thread, but at present I'm reading email as plain text files directly off the file system - which is a PITA ;-) ] Bjarne D Mathiesen København N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit miljø MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3 On a note regarding my hardware : the computer has three ethernet-interfaces: but that doesn't really seem to be an issue as telnet from localhost also hangs indefinetely I'm seeing the issue no matter which interface I connect to, whether the two external WAN or the internal LAN IP-number . root# system_profiler SPEthernetDataType 2012-01-22 01:34:36.524 system_profiler[47377:903] Found bsdName (0x2603) for AppleUSBEthernet 2012-01-22 01:34:36.626 system_profiler[47377:903] Found bsdName (0x2c03) for AppleUSBEthernet Ethernet Cards: Apple USB Ethernet Adaptor: Type: USB BSD name: en4 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8a:f3:5a Product ID: 5122 Vendor ID: 1452 Apple USB Ethernet Adaptor: Type: USB BSD name: en3 Kext name: AppleUSBEthernet.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleUSBEthernet.kext Version: 2.1.39 MAC Address: 00:1f:f3:8b:26:c1 Product ID: 5122 Vendor ID: 1452 Marvell Yukon Gigabit Adapter 88E8053 Singleport Copper SA: Name: ethernet Type: Ethernet Controller Bus: PCI Vendor ID: 0x11ab Device ID: 0x4362 Subsystem Vendor ID: 0x11ab Subsystem ID: 0x5321 Revision ID: 0x0022 Link Width: x1 BSD name: en0 Kext name: AppleYukon2.kext Location: /System/Library/Extensions/IONetworkingFamily.kext/Contents/PlugIns/AppleYukon2.kext Version: 3.2.1b1 ethernet: Type: Ethernet Controller Bus: PCI Slot: AirPort Vendor ID: 0x168c Device ID: 0x001c Subsystem Vendor ID: 0x106b Subsystem ID: 0x0086 Revision ID: 0x0001 Link Width: x1 BSD name: en1 Kext name: AirPortAtheros21.kext Location: /System/Library/Extensions/IO80211Family.kext/Contents/PlugIns/AirPortAtheros21.kext Version: 425.14.7 root# ifconfig lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 inet 127.0.0.1 netmask 0xff000000 gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280 stf0: flags=0<> mtu 1280 en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 00:16:cb:a8:7d:01 inet6 fe80::216:cbff:fea8:7d01%en0 prefixlen 64 scopeid 0x4 inet 192.168.1.37 netmask 0xffffff00 broadcast 192.168.1.255 media: autoselect (1000baseT <full-duplex,flow-control>) status: active en4: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 00:1f:f3:8a:f3:5a inet6 fe80::21f:f3ff:fe8a:f35a%en4 prefixlen 64 scopeid 0x5 inet 90.184.13.xxx netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX <full-duplex,flow-control>) status: active en3: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 00:1f:f3:8b:26:c1 inet6 fe80::21f:f3ff:fe8b:26c1%en3 prefixlen 64 scopeid 0x6 inet 90.184.12.yyy netmask 0xfffffe00 broadcast 90.184.13.zzz media: autoselect (100baseTX <full-duplex,flow-control>) status: active fw0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 2030 lladdr 00:17:f2:ff:fe:8e:e6:d2 media: autoselect <full-duplex> status: inactive en1: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500 ether 00:17:f2:52:62:9e media: autoselect (<unknown type>) status: inactive root# system_profiler SPHardwareDataType Hardware: Hardware Overview: Model Name: Mac mini Model Identifier: Macmini1,1 Processor Name: Intel Core Duo Processor Speed: 1,83 GHz Number Of Processors: 1 Total Number Of Cores: 2 L2 Cache: 2 MB Memory: 2 GB Bus Speed: 667 MHz Boot ROM Version: MM11.0055.B08 SMC Version (system): 1.3f4 root# system_profiler SPSoftwareDataType Software: System Software Overview: System Version: Mac OS X 10.6.8 (10K549) Kernel Version: Darwin 10.8.0 Boot Volume: X Boot Mode: Normal Computer Name: MiniWeb User Name: System Administrator (root) Secure Virtual Memory: Enabled 64-bit Kernel and Extensions: No Time since boot: 5 days 1:31 >On 19.2.2012, at 14.53, postmaster wrote: > >> Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: >sql(abuse(a)mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , >concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM >mailbox WHERE username =3D 'abuse(a)mathiesen.info' AND domain =3D >'mathiesen.info' AND active =3D 1 > >Probably won't help with your problem, but it's better if you don't >fetch "mail" field. Just fetch the home and set: > >mail_location = maildir:~/ > >This avoids maildir autodetection. That seemed to fix the autodetection :-) > >> # OS: Darwin 10.8.0 i386 hfs > >With HFS+ it's a good idea to set maildir_copy_with_hardlinks = no OK - set to no as advised > >> a list "" * >> * LIST (\HasNoChildren) "." "INBOX" >> a OK List completed. >> a select inbox >> Connection closed by foreign host. >> 4604ds1-ynoe:~ root# > >You mean the session hangs here until it's killed? Yes ... It does some of what it's supposed to do : the dovecot-uidlist.lock is set email is copied from new to cur, but then nothing more seems to happen :-( 4604ds1-ynoe:mail root# ls -l mathiesen.info/macintosh\(a)mathiesen.info/ total 40 drwx------ 2 5000 5000 4352 Feb 21 09:58 cur -rw------- 1 5000 5000 8119 Feb 21 10:28 dovecot-uidlist -rw------- 1 5000 5000 32 Feb 21 10:28 dovecot-uidlist.lock -rw------- 1 5000 5000 8 Feb 21 10:28 dovecot-uidvalidity -r--r--r-- 1 5000 5000 0 Feb 21 10:28 dovecot-uidvalidity.4f4363ac -rw------- 1 5000 5000 40 Feb 21 10:28 dovecot.index.log drwxr-xr-x 2 5000 5000 68 Feb 19 11:18 dovecot.rawlog drwx------ 2 5000 5000 68 Feb 21 09:58 new drwx------ 2 5000 5000 68 Feb 21 09:11 tmp 4604ds1-ynoe:mail root# >Try attaching gdb into it: > >gdb -p <pid of imap session> >bt full 4604ds1-ynoe:~ root# ps -axj | grep dovecot root 40655 1 40655 4d84750 0 Ss ?? 0:00.02 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 40657 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/anvil root 40658 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/log root 40660 40655 40655 4d84750 0 S ?? 0:00.06 dovecot/config _dovecot 40662 40655 40655 4d84750 0 S ?? 0:00.02 dovecot/auth root 40663 40655 40655 4d84750 0 S ?? 0:00.03 dovecot/auth -w 5000 40677 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40689 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 5000 40803 40655 40655 4d84750 0 S ?? 0:00.01 dovecot/pop3 root 40618 362 40618 5680af8 1 S+ s001 0:00.01 tail -f dovecot.log root 40952 396 40951 4d85d40 2 R+ s003 0:00.00 grep dovecot 4604ds1-ynoe:~ root# gdb -p 40803 GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin". /private/var/root/40803: No such file or directory Attaching to process 40803. Reading symbols for shared libraries . done Reading symbols for shared libraries ...... done 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001f35 in login_client_connected () No symbol table info available. #22 0x000eb1a8 in master_login_auth_finish () No symbol table info available. #23 0x000eae80 in master_login_auth_callback () No symbol table info available. #24 0x000ebb3a in master_login_auth_input () No symbol table info available. #25 0x001001a7 in io_loop_call_io () No symbol table info available. #26 0x00101492 in io_loop_handler_run () No symbol table info available. #27 0x00100148 in io_loop_run () No symbol table info available. #28 0x000ed25a in master_service_run () No symbol table info available. #29 0x00001df8 in main () No symbol table info available. (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from process 40803. 4604ds1-ynoe:~ root# > >or alternatively try: > >gdb --args /usr/local/libexec/dovecot/imap -u user@domain >run >a select inbox ><ctrl-c> >bt full 4604ds1-ynoe:~ root# gdb --args /macports/libexec/dovecot/pop3 -u postmaster(a)mathiesen.info GNU gdb 6.3.50-20050815 (Apple version gdb-1518) (Sat Feb 12 02:49:13 UTC 2011) Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries ..... done (gdb) run Starting program: /macports/libexec/dovecot/pop3 -u postmaster(a)mathiesen.info Reading symbols for shared libraries ++++. done Debug: auth input: postmaster(a)mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/ Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/ Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/, alt= Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/postmaster(a)mathiesen.info/: mode=0700 gid=-1 a select inbox ^C Program received signal SIGINT, Interrupt. 0x90003afa in mach_msg_trap () (gdb) bt full #0 0x90003afa in mach_msg_trap () No symbol table info available. #1 0x90004267 in mach_msg () No symbol table info available. #2 0x9001b431 in vproc_mig_look_up2 () No symbol table info available. #3 0x9001b26a in bootstrap_look_up2 () No symbol table info available. #4 0x9001b0e4 in bootstrap_look_up () No symbol table info available. #5 0x9001af30 in notify_register_check () No symbol table info available. #6 0x9001ada7 in notify_register_tz () No symbol table info available. #7 0x9001a621 in tzsetwall_basic () No symbol table info available. #8 0x90041903 in localtime () No symbol table info available. #9 0x0007ba2f in mail_index_update_day_headers () No symbol table info available. #10 0x00077c95 in mail_index_transaction_finish () No symbol table info available. #11 0x0007680a in mail_index_transaction_commit_v () No symbol table info available. #12 0x00076acf in mail_index_transaction_commit_full () No symbol table info available. #13 0x00076b18 in mail_index_transaction_commit () No symbol table info available. #14 0x0007f434 in mail_index_sync_commit () No symbol table info available. #15 0x0004ff02 in maildir_sync_index_finish () No symbol table info available. #16 0x00051376 in maildir_sync_context () No symbol table info available. #17 0x000516f2 in maildir_storage_sync_init () No symbol table info available. #18 0x000125fd in mailbox_sync () No symbol table info available. #19 0x000024ba in client_create () No symbol table info available. #20 0x00001a2a in client_create_from_input () No symbol table info available. #21 0x00001d71 in main () No symbol table info available. (gdb) quit The program is running. Exit anyway? (y or n) y 4604ds1-ynoe:~ root#

2 1

[Dovecot] Folder name migration when moving from 2.0.x to 2.1.x?
by Ralf Hildebrandt 21 Feb '12

21 Feb '12

Today I encountered this after upgrading: Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Panic: LIST: Mailbox name not UTF-8: INBOX.Entw�rfe Feb 18 18:15:40 postamt dovecot: imap(lehnertr): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x4104a) [0xb75cc04a] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x410bb) [0xb75cc0bb] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb75cc9b7] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8052bac] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_list_full+0x4af) [0x80530df] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](cmd_lsub+0x17) [0x8053537] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](command_exec+0x32) [0x8057c92] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056c7d] -> dovecot/imap [lehnertr 85.178.5.225 LSUB]() [0x8056d95] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_handle_input+0x135) [0x8056fa5] -> dovecot/imap [lehnertr 85.178.5.225 LSUB](client_input+0x60) [0x80578a0] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_call_io+0x44) [0xb75dafb4] -> /usr/local/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0xce) [0xb75dbe5e] -> /usr/local/lib/dovecot/libdovecot.so.0 a) It should not PANIC (I think) b) How can I globally "fix" the folder names (preferably in such a way that I am able to switch back and forth between 2.0 and 2.1) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

5 13

[Dovecot] 2.1 upgrade docs?
by Marc Perkel 21 Feb '12

21 Feb '12

Just wondering if there's any upgrade docs for 2.1 online?

7 7

[Dovecot] dovecot 2 doesn't start with managesieve
by nti＠w4w.net 21 Feb '12

21 Feb '12

Hi I build dovecot 2.0.16 and and dovecot-pigeonhole-0.2.5 on two machines with the same options. The config files are equivalent. On the first machine everything works fine - on the second dovecot with managesive won't start (without sieve protocol it does). Error: socket() failed: Protocol not supported Error: service(managesieve-login): listen(::, 4190) failed: Protocol not supported Fatal: Failed to start listeners I controlled the config, build pigeonhole a second time - nothing helps. Any idea ? Is there a possibility to debug the start? Nicola ... plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap pop3 sieve ... service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } service managesieve { process_limit = 1024 } ... protocol sieve { managesieve_max_compile_errors = 10 }

2 1

[Dovecot] user_filter problem
by sanito＠progresas.lt 21 Feb '12

21 Feb '12

Hello, I have a little bit complex setup. I'm trying to use dovecot lmtp and pop3 with user information stored in the LDAP database. The main problem is that email address and pop3 username are different for every mailbox (I cannot change this behaviour, because thousands of people already configured their email client for such a setup). So in my situation, user_filter for lmtp should be: user_filter=(mail=%u), but for pop3 it should be: user_filter=(cn=%u). Please, see: E-mail address: john(a)example.com pop3 username: 0005a pop3 password: very_secure_password And ldap ldiff for this mailbox: dn: cn=0005a,dc=example.com,ou=localmail,ou=mail uid: 0005a mail: john(a)example.com mailbox: /data/mail/example.com/0005a/Maildir/ sn: 0005a description: testing homeDirectory: /data/mail/example.com/0005a cn: 0005a userPassword:: e0NSWVBUfWxpOXBVaEw5SWJkRWc= objectClass: inetOrgPerson objectClass: CourierMailAccount objectClass: top I think I need to use different user_filter for lmtp and pop3, but it's not possible or I cannot find how to do it. Please, help! -- Best regards, dammit

2 1

[Dovecot] Possible broken indexer(lucene/solr)?
by Ingo Thierack 21 Feb '12

21 Feb '12

Hello, try to use the new 2.1rc and don't get any data in the searchindex. Tried first lucene, and switched than back so solr. If I do an search in an mailfolder, i get in the dovecot-log 2012-02-06 22:17:11 | dovecot: | indexer-worker(xxxxxx): Indexed 0 messages in INBOX/dovecot Log from solr. Feb 6, 2012 10:17:11 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=uid,score&sort=uid+asc&fq=%2Bbox:120ed10bbe9dcd4c8d2e0000f8146a47+%2Buser:"xxxxxxx"&q=body:"solr"&rows=9159} hits=0 status=0 QTime=1 Maybe I miss something. Upgraded from 2.0.15 to 2.1(head from repository yesterday) With 2.0 i see, if i start an search, solr had to work on the mail, now happens nothing. Upgrade the schema.xml, delete the old index. Regard Ingo Thierack

4 4

[Dovecot] Feature request for maillog plugin
by Aydın Demirel 21 Feb '12

21 Feb '12

Hi, Is it possible to add the modification time and file name of the mail file to maillog plugin? Thanks -- *Aydın Demirel Endersys Ltd. Sistem Destek Mühendisi/ System Support Engineer* * *<<Endersys is the first Advanced Business Partner of Red Hat in Turkey! http://www.europe.redhat.com/partners/endersys/>> Phone : +90 216 470 9423 | GSM : +90 530 401 8203 Fax : +90 216 470 9508 | Web : http://www.endersys.com <http://www.endersys.com/> Blog : http://blog.endersys.com <http://blog.endersys.com/> Twitter : http://www.twitter.com/endersys LPI : The #1 Linux Certification for IT Professionals LPI (Linux Professional Institute) Turkey http://www.lpi-turkey.com <http://www.lpi-turkey.com/>

2 1

[Dovecot] Homedir vs locations vs mail_location?
by Alexander Chekalin 21 Feb '12

21 Feb '12

Hello, I use Dovecot for rather long time, but I'm still in doubt for some small things. Here they are: 1. The homedir value points to the place where everything for the user stored at, while mail_location is something (some place) where mail stored at. if I deal with pure virtual users (all users are in sql tables and no system homes for them at all), should I ever care for returning meaningful value for 'homedir' (via password_query's userdb_home), or I can simple return empty or constant ('' or '123') for it and it won't mess anything? 2. If I use single (default) namespace, should I set namespace's location (to the same value as global mail_location), and should I expect anything strange if I skip it to set? Reversely, is it possible not to set global mail_location and set only namespace's location (which would be more logical as namespace definition is compact and easy to find in config)? I've reread my questions and see it is dummy ones, but I'd like to know that for sure. Thank you in advance, Alexander

3 2

[Dovecot] dovecot freezes when trying to get mail from maildir with mail
by postmaster 21 Feb '12

21 Feb '12

I'm completely stumped by this problem and I've tried everything at present - including installing dovecot from scratch. The problem is that when trying to get email from the server -AND- there's email in the account, the pop3 or imap process freezes and no email is fetched. The crazy thing is, that it has been working perfectly, but suddenly I'm getting these freezes both when using telnet from localhost, roundcubemail on the same server and Seamonkey from another computer. Logging seems to work perfectly. For an account without email, dovecot finds the maildir++ and acts correctly. For an account -with- email, dovecot seems to find the maildir++, but also seems to try Namespace of which I haven't defined any. I've tried to get rawlog to work also, but that didn't work - the telnet process froze on login even though I'm quite sure I did the right thing and ps showed an active rawlog process Postfix is delivering the email straight into the mailboxes. Here is the information 4604ds1-ynoe:~ root# port installed | grep dovecot dovecot2 @2.0.18_0+mysql5+postgresql91 (active) dovecot2-sieve @0.2.5_1 (active) postfix @2.9.0_0+dovecot_sasl+mysql5+pcre+postgresql91 (active) 4604ds1-ynoe:~ root# 4604ds1-ynoe:log root# cat dovecot.log.txt Feb 19 11:46:52 4604ds1-ynoe dovecot[441]: master: Dovecot v2.0.18 starting up (core dumps disabled) Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:51:47 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=495) Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49180 resp=AGFidXNlQG1hdGhpZXNlbi5pbmZvAHRlc3R0ZXN0 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: Loading modules from directory: /macports/lib/dovecot/auth Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: mysql(/macports/var/run/mysql5/mysqld.sock): Connected to database infomathiesenpostfixadmin Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse(a)mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'abuse(a)mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=abuse(a)mathiesen.info Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 738852865 495 1 687658e957c0e4f473891548eecc49be Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(abuse(a)mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'abuse(a)mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 738852865 abuse(a)mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap-login: Login: user=<abuse(a)mathiesen.info>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=503, secured Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse(a)mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info/ Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse(a)mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info/storage) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse(a)mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:52:20 4604ds1-ynoe dovecot[443]: imap(abuse(a)mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/abuse(a)mathiesen.info, alt= Feb 19 11:53:01 4604ds1-ynoe dovecot[443]: imap(abuse(a)mathiesen.info): Disconnected: Logged out bytes=45/750 Feb 19 11:53:06 4604ds1-ynoe dovecot[443]: auth: Debug: auth client connected (pid=512) Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client in: AUTH 1 PLAIN service=imap secured lip=127.0.0.1 rip=127.0.0.1 lport=143 rport=49182 resp=AG1hY2ludG9zaEBtYXRoaWVzZW4uaW5mbwB0ZXN0dGVzdA== Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh(a)mathiesen.info,127.0.0.1): query: SELECT username AS user, domain, password FROM mailbox WHERE username = 'macintosh(a)mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: client out: OK 1 user=macintosh(a)mathiesen.info Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master in: REQUEST 1866334209 512 1 332714ca2a070d7adb342d4baba4a19a Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth-worker: Debug: sql(macintosh(a)mathiesen.info,127.0.0.1): SELECT 5000 AS uid , 5000 AS gid , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS home , concat ( '/Volumes/BrugerData/PostFix/mail/' , maildir ) AS mail FROM mailbox WHERE username = 'macintosh(a)mathiesen.info' AND domain = 'mathiesen.info' AND active = 1 Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: auth: Debug: master out: USER 1866334209 macintosh(a)mathiesen.info uid=5000 gid=5000 home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info/ mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap-login: Login: user=<macintosh(a)mathiesen.info>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=522, secured Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap: Debug: Added userdb setting: mail=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh(a)mathiesen.info): Debug: Effective uid=5000, gid=5000, home=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info/ Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh(a)mathiesen.info): Debug: mdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info/storage) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh(a)mathiesen.info): Debug: sdbox autodetect: stat(/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info/mailboxes) failed: No such file or directory Feb 19 11:53:55 4604ds1-ynoe dovecot[443]: imap(macintosh(a)mathiesen.info): Debug: maildir++: root=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info, index=, control=, inbox=/Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info, alt= Feb 19 11:54:13 4604ds1-ynoe dovecot[443]: imap(macintosh(a)mathiesen.info): Debug: Namespace : Using permissions from /Volumes/BrugerData/PostFix/mail/mathiesen.info/macintosh(a)mathiesen.info: mode=0700 gid=-1 Feb 19 12:12:16 4604ds1-ynoe dovecot[441]: master: Error: service(imap): child 522 killed with signal 3 Feb 19 12:15:48 4604ds1-ynoe dovecot[441]: master: Warning: Killed with signal 15 (by pid=1086 uid=0 code=unknown 0) 4604ds1-ynoe:log root# 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 441 1 441 56804e0 0 Ss ?? 0:00.01 /macports/sbin/dovecot -c /macports/etc/dovecot/dovecot.conf _dovecot 442 441 441 56804e0 0 S ?? 0:00.01 dovecot/anvil root 443 441 441 56804e0 0 S ?? 0:00.01 dovecot/log 5000 522 441 441 56804e0 0 S ?? 0:00.01 dovecot/imap root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 543 380 542 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# kill -QUIT 522 4604ds1-ynoe:mail root# dovecot stop 4604ds1-ynoe:mail root# ps -axj | grep dovecot root 452 362 452 5680af8 1 S+ s001 0:00.00 tail -f dovecot.log root 1091 380 1090 4d84c30 2 R+ s002 0:00.00 grep dovecot 4604ds1-ynoe:mail root# 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "abuse(a)mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1326334461] UIDs valid * OK [UIDNEXT 4] Predicted next UID * OK [HIGHESTMODSEQ 1] Highest a OK [READ-WRITE] Select completed. a logout * BYE Logging out a OK Logout completed. Connection closed by foreign host. 4604ds1-ynoe:~ root# telnet localhost 143 Trying 127.0.0.1... Connected to localhost (127.0.0.1). Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. a login "macintosh(a)mathiesen.info" "x" a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS] Logged in a list "" * * LIST (\HasNoChildren) "." "INBOX" a OK List completed. a select inbox Connection closed by foreign host. 4604ds1-ynoe:~ root# 4604ds1-ynoe:~ root# dovecot -n # 2.0.18: /macports/etc/dovecot/dovecot.conf # OS: Darwin 10.8.0 i386 hfs auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain base_dir = /macports/var/run/dovecot/ default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 mail_debug = yes mail_gid = 5000 mail_location = maildir:/Volumes/BrugerData/PostFix/mail/%d/%u mail_uid = 5000 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mmap_disable = yes passdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size } protocols = imap pop3 service postlogin { executable = script-login -d rawlog } ssl = no syslog_facility = local2 userdb { args = /macports/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol pop3 { pop3_client_workarounds = oe-ns-eoh outlook-no-nuls } 4604ds1-ynoe:~ root#

2 1

[Dovecot] Something is strange - perhaps my AMD processors ?
by Finn B 21 Feb '12

21 Feb '12

Moi Timo / all. I've been using Dovecot on my Qmailtoaster production mailserver for some years - since 1.0.7 and current Version 2.0.10 - works perfectly. Now I am installing a backupsystem and a VM testsystem (2 boxes) - and no matter how and what Version I am installing on these 2 boxes I get below messages whenever I try to test connection with Telnet (localhost 110 / 995). I have tried to install all Dovecot versions from 2.010 to 2.018 as rpms, yums and as source and compiled it. It's on Centos5.7 (64) with Qmailtoaster (which I have installed as I have done many times before) I have tried to copy all my settings from my current production system (Centos 5.7, latest and greatest versions of qmailtoaster and Dovecot 2.0.10) - nothing helps. Now I come to think of it I realize that my production system is intel based and the 2 new ones are AMD based and wonder if this may be the case ? Dovecot.log master: Error: Service(auth): command startup failed, throttling log: Error: Service(auth): child 1859 returned error 89 (Fatal failure) pop3-login: Error: Timeout waiting for handshake from auth server. my pid=1858, input bytes=0 Regards, Finn B

7 8

[Dovecot] Questions still about a uw-imap -> dovecot conversion
by Steve Campbell 20 Feb '12

20 Feb '12

The more I read about all of this, the more I'm thinking about moving to maildir format. My switchover this weekend is full of holes due to the way user's imap folders were laid out. Some had folders in their home directory and others might have folders in their /home/mail directory. Some how, Horde/Imp seemed to keep track of it. Dovecot is not doing so well. I've been reading about the conversion technique to convert to maildir, but I'm wondering if I can do this based on the way our users are set up. Most are pop accounts. Our webmail is configured to read the inbox from /var/spool/mail and their imap folders from /home or /home/mail. Some have imap accounts on their desktop defined through their mail client. RH/Centos seems to want me to switch to postfix since most of the documentation I find doesn't mention Sendmail. My filesystem is laid out to handle Inboxes in /var/spool/mail and imap folders in /home(/mail) and these aren't logical volumes but true partitions, so I've got to consider this as I make any changes. I also run pop using dovecot, so I'll need to figure all this out. I've read the technique linked to on the dovecot site that points to "Replacing UW-IMAP with Dovecot on RHEL 3" written back in 2004, and that's about the only real pointers I'm finding. I don't want to have to go around to each machine and change their clients to IMAP.. Anyone care to comment on anything similar they've done and how many pitfalls they ran into and what they were? steve campbell

3 4

[Dovecot] A Postfix/Dovecot example with SQLite backend [crossposted]
by /dev/rob0 19 Feb '12

19 Feb '12

There are many mail howtos on the web ... can one more hurt? http://rob0.nodns4.us/howto/README http://rob0.nodns4.us/howto/ http://rob0.nodns4.us/howto/latest.tar.gz (all files) (Sorry, not HTML yet. That is on the agenda.) This is a multiple address class sample implementation of a Postfix MTA and Dovecot IMAP server using a SQLite3 data backend. Domain lookups, user maps, access and transport maps: all using a single, shared SQLite database file. What, other than the SQLite backend, distinguishes this from other mail system howtos? The Postfix high points include a complete implementation of all address classes and per-address transport(5) maps, virtual(8) UID/GID maps, and smtpd(8) recipient access(5) maps. (The latter is using smtpd_restriction_classes, which are not discussed in detail, but are implemented in an interesting way.) On the Dovecot side, it's mostly standard stuff. The SQL deny userdb implementation, and the seamless integration of system and SQL users, might be interesting. I think the database itself is the best part of this example. It's as close to "normalized" as I think it can reasonably be. A significant fact is that each revision of the system has tended to simplify the schema. That's a good sign, I think. One central Domain table lists all domains and hostnames to which the server makes reference. Likewise, a central Address table lists all addresses (with a pointer to the Domain table for each record.) The Alias table defines relationships between Address entries. (Both local(8) and virtual(5) alias maps exist in that table.) Comments and suggestions are welcome, on-list if it's topical to whichever list (please don't crosspost unless comments are relevant to both lists), or offlist to the address in the README file (or as detailed below.) Thanks for your interest. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if "/dev/rob0" is in the Subject:

1 0

[Dovecot] Automatic Debian packages for Dovecot v2.2.
by Stephan Bosch 19 Feb '12

19 Feb '12

Hi, For our testing enthusiasts I've created new automatic Debian repositories for Dovecot v2.2. Repositories are available for Debian oldstable, stable and testing, on both i386 and am64. Read the following wiki item for more information: http://wiki2.dovecot.org/PrebuiltBinaries#Automatically_Built_Packages The repository for Dovecot v1.2 has been defunct for a few months now and is expected to be removed soon. For the coming month or so, the builders at Xi will not be active during Dutch daytime. There are some issues with the virtual host's I/O performance that need to be resolved first. Regards, Stephan.

2 1

[Dovecot] I'm confused about my namespace parms
by Steve Campbell 19 Feb '12

19 Feb '12

Hi, I'm new to the list and to dovecot. I'm having a strange situation that provides no errors, so I'm having a little problem diagnosing what's going on. I've been running a Centos 3 mail server with sendmail for our company for quite a few years. I finally got approval for new servers this year and put Centos 6.2 on it. I'm sticking with sendmail, but that's not important. Up to this point, I've got all working fairly well. I've found that if I configure dovecot to use the mixed format (pop and imap INBOX in /var/spool/mail, Imap folders in /home/user/mail, two namespaces) as in the examples for "namespace", most works fine. Anyone who has used imap previously on our old system and work as if nothing has changed. Anyone who has never used Imap before can't seem to access the server using our webmail app (Horde/Imp). Dovecot is providing both pop and imap. An account that is set up from a normal mail client as imap has no problems as well. Sendmail is set up to create a new user in the /var/spool/mail folder. When I moved from the old server to the new server, I copied /home and /var/spool/mail to the new server - in other words, I kept the same scheme. There are no errors in the maillog, and it indicates that the user is getting logged in and out, but the return to Horde is that the user entered the wrong name or password. I'm thinking that this is probably due to how dovecot is trying to look for the imap folders, but the lack or any errors makes it a little difficult to diagnose. I'm fairly certain that what was installed on the old server was UW-Imap and was provided by the imap-2002 rpm. I tried using the examples for UW-Imap compatability but that didn't work at all. I haven't been able to figure out what's different about the old imap folder contents and what's missing in the newly created imap folders to allow Horde to access this account. I'll look at the Horde config to see if anything is there, but I'm cautious about changing anything that might break the current users' ability. Most of the accounts are set up as POP with the ability to see what's left on the server using Horde, webmail, and IMAP. Any help in tracing this down would really help. I think my biggest problem is I don't understand the "namespace" parameter as well as I should. So there it is. It seems I've got at least 3 areas to concentrate on. Steve Campbell ------------------------------------------------- This mail sent through IMP: http://horde.org/imp/

2 2

[Dovecot] Move templates args to override_fields setting?
by Ralf Hildebrandt 18 Feb '12

18 Feb '12

After upgrading from 2.0.x to 2.1.x I get a lot of: Feb 18 18:18:12 postamt dovecot: auth-worker(31443): Warning: userdb passwd: Move templates args to override_fields setting OK, it's just a warning, but how do I fix that in a 2.0.x compatible way? The only place userdb is used is in this contaxt: # for user*masteruser logins passdb { args = /usr/local/etc/dovecot/dovecot.masteruser driver = passwd-file master = yes } # User via passwd userdb { driver = passwd args = cache_key=%u } -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

2 2

[Dovecot] expunging
by Micah Anderson 18 Feb '12

18 Feb '12

I am having trouble scaling some regular dovecot cleanup operations on our servers. On a daily basis, I'm wanting to do this on each server, which contains its own isolated set of user storage: /usr/bin/doveadm expunge -A mailbox Trash* savedbefore 21d /usr/bin/doveadm expunge -A mailbox Spam savedbefore 7d /usr/bin/doveadm expunge -A mailbox Sent savedbefore 120d but these are a very expensive operations. For example, just doing the Spam expunge takes 30 minutes (or more, depending on the load) of heavy disk operations, on each machine it is run on. There are approximately 20k users on each machine. It seems like it does not use the iterate query, but rather it looks into the database at the expires table and rather it iterates over every user mentioned there. This is a problem because I've got multiple dovecot machines with different sets of users on them, using the same table, so that means that its doing stat() calls on each system for every user mentioned there and failing to find the user on the filesystem (because the user is on another system). In these cases it spits out an error: doveadm(user): Info: User no longer exists, skipping For users that do exist on this system, it seems to do something on the order of 15 stat() calls, at minimum. What are some things I can do to make this less of an expensive process? If I had a shared storage system that each machine used, this would cut down on the resource waste because I'd only need to run the query once, but unfortuntely, that isn't how these systems were designed. thanks for any ideas, tips etc. micah

3 3

[Dovecot] Quota warnings and LMTP
by joshua＠hybrid.pl 18 Feb '12

18 Feb '12

Hello, I have dovecot 2.0 configured with virtual users in MySQL, SMTP is handled by exim which uses dovecot LMTP transport. I'd like to test quota warning messages, but in wiki2 there is only an example for delivering it with dovecot-lda. I tried several ways of using lmtp binary, but without success. How to replace this: cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" From: postmaster(a)domain.com Subject: quota warning Your mailbox is now $PERCENT% full. EOF with lmtp? Greetings, -- Jacek Osiecki

3 2

[Dovecot] Order of operations in Dovecot
by Eric Gregory 18 Feb '12

18 Feb '12

Hi, I'm working on an IMAP client and had a question about the order in which commands are responded to in Dovecot. In other IMAP servers I've looked at, when you issue two commands simultaneously (i.e. pipelining) it looks something like this: a fetch something b fetch something else * 1 FETCH (result) a OK FETCH completed * 1 FETCH (result) b OK FETCH completed This works well because it's clear which results are associated with which tag; the 'a' fetch results immediately precede the a's OK response, and likewise the 'b' fetch results immediately precede the b's OK response. But in Dovecot (version 2.0.13) the results of the two simultaneous commands are ordered like this: a fetch something b fetch something else * 1 FETCH (result) * 1 FETCH (result) a OK FETCH completed b OK FETCH completed This makes associating the results of the fetch commands quite a bit more difficult, since it's not entirely clear which results are paired with which tag. My question is this: is this Dovecot's intended behavior, or did I find a bug? I realize the IMAP spec is really vague about this. If this is the intended behavior, is there any advice for clients on how to determine which results are associated with which command? Thanks! - Eric

2 1

[Dovecot] Upgraded to 2.1 - ulimit error
by Marc Perkel 17 Feb '12

17 Feb '12

Not sure what this means: Warning: fd limit (ulimit -n) is lower than required under max. load (4096 < 20000), because of service auth { client_limit } What do I need to do?

1 1

[Dovecot] Dovecot 2.1.0 packages for CentOS 6 / RHEL6
by Morten Stevens 17 Feb '12

17 Feb '12

Hi, First, Timo, Thank you for Dovecot 2.1.0. I've built Dovecot 2.1.0 RPMs including Dovecot-Pigeonhole 0.3.0 for RHEL6, CentOS 6 and Scientific Linux 6. If anyone is interested in testing these RPMs: http://oss.imt-systems.com/mstevens/el6/dovecot/2.1.0/ Feedback would be appreciated. After some testing, we will use these RPMs on our IMAP servers. Best regards, Morten

2 2

[Dovecot] Dovecot lda parameters
by Клименко Николай 17 Feb '12

17 Feb '12

hi all I have the following scheme. Postfix with virtual users with dovecot LDA. AD as database. Dovecot and Postfix get the following Idap attributes from AD: sAMAccountName, mail and mail_alias. The problem is that I can’t make dovecot lda move the user’s mail to the folder with the name from sAMAccountName attribute, as mail always moves to folder with e-mail address name. Is it posible to delivery e-mail to folder with name from ldap attributes sAMAccountName? --

1 0

[Dovecot] Favorable mention in Linux Productivity Magazine
by Steve Litt 17 Feb '12

17 Feb '12

Hi all, The latest Linux Productivity Magazine, http://www.troubleshooters.com/lpm/201202/201202.htm, discusses the positive role Dovecot played in my migration away from Kmail. This is currently topical as Kmail is in the midst to transforming to Kmail2, which requires an Akonadi database and is not popular. I hope you enjoy the magazine. SteveT Steve Litt Troubleshooters.Com

1 0

[Dovecot] How to delete old mails in dovecot 1.2.x
by Przemysław Orzechowski 17 Feb '12

17 Feb '12

Hi I have already read this page but there is no info on how to delete old mails in dovecot 1.2 http://wiki1.dovecot.org/QuestionsAndAnswers#How_can_I_create_a_cron_or_som… I need to delete old mails from over 100 mailboxes with average of 10k mails / mailbox Is there a way to delete old mails (by message date not file creation date) as with doveadm in dovecot 2 ? Thanks in advance bellow my dovecot -n (not sure if it is neccessary or not so just in case) # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-33-server x86_64 Ubuntu 10.04.3 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s ssl_cert_file: /etc/ssl/makolab/star_makolab_pl.crt ssl_key_file: /etc/ssl/makolab/star_makolab_pl.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_location: maildir:~/Maildir mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 auth default: passdb: driver: pam userdb: driver: passwd plugin: quota: fs:Limit uzytkownika:user quota2: fs:Limit domeny:group

3 4

[Dovecot] log message about client_limit may in fact be due to ulimit
by Christoph Bußenius 17 Feb '12

17 Feb '12

Hi, our Dovecot proxy started logging about reaching the client_limit once there were about 1000 connections running over it: Feb 14 15:22:02 vmmailproxy1 dovecot: master: Warning: service(anvil): client_limit (100000) reached, client connections are being dropped Feb 14 15:22:18 vmmailproxy1 dovecot: master: Warning: service(ipc): client_limit (100000) reached, client connections are being dropped I was a bit perplexed because 100.000 is far above 1000. However we found out that in fact the problem was the number of open files, and we have fixed it by putting 'ulimit -n 6000' into the init script. Even though the log claims that connections would be dropped, the number of imap-login processes kept rising above 1000. The "doveadm proxy list" would fail with an error message: $ sudo doveadm proxy list doveadm(root): Error: LIST failed: Disconnected Segmentation fault The segfault did not happen every time I tried it. I'm sorry I don't have any core dumps or backtraces, because I was more concerned about putting the proxy into a sane state. Maybe I can reproduce this on a test server if you would like a back trace. Anyway, I suggest changing the log message if the failure is not actually due to the client_limit. # 2.0.18: /etc/ports/dovecot/dovecot.conf # OS: Linux 2.6.32-38-server x86_64 Ubuntu 10.04.2 LTS auth_username_format = %n default_client_limit = 60000 default_process_limit = 10000 disable_plaintext_auth = no mail_location = maildir:/nonexistent managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = /etc/ports/dovecot/passwd driver = passwd-file } protocols = imap pop3 lmtp sieve service anvil { client_limit = 100000 } service auth { client_limit = 60000 } service ipc { client_limit = 100000 } service managesieve-login { inet_listener sieve { port = 4190 } inet_listener sieve_deprecated { port = 2000 } } shutdown_clients = no ssl_cert = </etc/ssl/certs/mailproxy+chain.pem ssl_key = </etc/ssl/private/mailproxy.key syslog_facility = local6 verbose_proctitle = yes protocol imap { imap_capability = IMAP4rev1 IDLE THREAD=REFERENCES SORT QUOTA ACL UIDPLUS } The passwd file has all entries looking like this: login:::::::proxy=y host=backend1 nopassword=yes -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 <> Raum 00.05.055 <> Boltzmannstr. 3 <> Garching

2 6

[Dovecot] Released Pigeonhole v0.3.0 for Dovecot v2.1.0
by Stephan Bosch 17 Feb '12

17 Feb '12

Hello Dovecot users, Now that Dovecot v2.1.0 is out there, I release Pigeonhole v0.3.0. The most significant new features are support for the Sieve editheader extension and the sieve-filter tool. I haven't received much feedback on these new features, which can either mean that they are mostly bug-free or that not many people have tried these, so these are still considered experimental. Changelog v0.3.0: * Renamed sieve_global_path setting to sieve_default for clarity. Old name is still recognized for backwards compatibility. Support for the ancient (pre v1.1) name for this setting "global_script_path" is now dropped. * Added means to prohibit use of redirect action. Setting sieve_max_redirects to 0 now means that redirect is disallowed in stead of unlimited. Default value remains four. * Fixed interaction of Sieve include extension with ManageSieve. It is updated to match new requirements in the draft include specification. Missing included scripts are no longer an error at upload time. * Updated RFC2822 header field body verification to exclude non-printing characters (RFC5322). Only Sieve actions that can create unstructured header values (currently enotify/mailto and editheader) are affected by this change. + Completed sieve-filter tool to a useful state. The sieve-filter tool provides a means to (re)filter messages in a mailbox through a Sieve script. + Implemented the Sieve editheader extension. It is now possible to add and remove message headers from within Sieve. + ManageSieve: added support for reading quoted and literal strings as a stream. Fixes support for handing large SASL responses (analogous to similar changes in Dovecot). It is now also allowed to use a quoted string for the PUTSCRIPT script argument. + Added code to cleanup tmp directory in Sieve storage directory (sieve_dir) every once in a while. + Added support for substituting the entire message during Sieve processing. This is used for the filter action provided by the new sieve_extprograms plugin (provided separately for now). The filter action allows passing the message through an external program. + Added support for restricting certain Sieve language extensions to (admin-controled) global scripts. Restricted extensions can be configured using the new sieve_global_extensions setting. This is particularly useful for some of the Dovecot-specific (plugin-based) Sieve extensions, that can be somewhat hazardous when under direct control of users (e.g. sieve_extprograms). The release is available as follows: http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz http://www.rename-it.nl/dovecot/2.0/dovecot-2.1-pigeonhole-0.3.0.tar.gz.sig Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

2 2

[Dovecot] Dovecot v2.2 plans
by Timo Sirainen 16 Feb '12

16 Feb '12

Here's a list of things I've been thinking about implementing for Dovecot v2.2. Probably not all of them will make it, but I'm at least interested in working on these if I have time. Previously I've mostly been working on things that different companies were paying me to work on. This is the first time I have my own company, but the prioritization still works pretty much the same way: - 1. priority: If your company is highly interested in getting something implemented, we can do it as a project via my company. This guarantees that you'll get the feature implemented in a way that integrates well into your system. - 2. priority: Companies who have bought Dovecot support contract can let me know what they're interested in getting implemented. It's not a guarantee that it gets implemented, but it does affect my priorities. :) - 3. priority: Things other people want to get implemented. There are also a lot of other things I have to spend my time on, which are before the 2. priority above. I guess we'll see how things work out. dsync-based replication ----------------------- I'll write a separate post about this later. Besides, it's coming for Dovecot v2.1 so it's a bit off topic, but I thought I'd mention it anyway. Shared mailbox improvements --------------------------- Support for private flags for all mailbox formats: namespace { type = public prefix = Public/ mail_location = mdbox:/var/vmail/public:PVTINDEX=~/mdbox/indexes-public } - dsync needs to be able to replicate the private flags as well as shared flags. - might as well add a common way for all mailbox formats to specify which flags are shared and which aren't. $controldir/dovecot-flags would say which is the default (private or shared) and what flags/keywords are the opposite. - easy way to configure shared mailboxes to be accessed via imapc backend, which would allow easy shared mailbox accesses across servers or simply between two system users in same server. (this may be tricky to dsync.) - global ACLs read from a single file supporting wildcards, instead of multiple different files - default ACLs for each namespace/storage root (maybe implemented using the above..) Metadata / annotations ---------------------- Add support for server, mailbox and mail annotations. These need to be dsyncable, so their changes need to be stored in various .log files: 1. Per-server metadata. This is similar to subscriptions: Add changes to dovecot.mailbox.log file, with each entry name a hash of the metadata key that was changed. 2. Per-mailbox metadata. Changes to this belong inside mailbox_transaction_context, which write the changes to mailbox's dovecot.index.log files. Each log record contains a list of changed annotation keys. This gives each change a modseq, and also allows easily finding out what changes other clients have done, so if a client has done ENABLE METADATA Dovecot can easily push metadata changes to client by only reading the dovecot.index.log file. 3. Per-mail metadata. This is pretty much equivalent to per-mailbox metadata, except changes are associated to specific message UIDs. The permanent storage is in dict. The dict keys have components: - priv/ vs. shared/ for specifying private vs. shared metadata - server/ vs mailbox/<mailbox guid>/ vs. mail/<mailbox guid>/<uid> - the metadata key name This would be a good time to improve the dict configuration to allow things like: - mixed backends for different hierarchies (e.g. priv/mailbox/* goes to a file, while the rest goes to sql) - allow sql dict to be used in more relational way, so that mail annotations could be stored with tables: mailbox (id, guid) and mail_annotation (mailbox_id, key, value), i.e. avoid duplicating the guid everywhere. Things to think through: - How to handle quota? Probably needs to be different from regular mail quota. Probably some per-user "metadata quota bytes" counter/limit. - Dict lookups should be done asynchronously and prefetched as much as possible. For per-mail annotation lookups mail_alloc() needs to include a list of annotations that are wanted. Configuration ------------- Copy all mail settings to namespaces, so it'll be possible to use per-namespace mailbox settings. Especially important for imapc_* settings, but can be useful for others as well. Those settings that aren't explicitly defined in the namespace will use the global defaults. (Should doveconf -a show all of these values, or simply the explicitly set values?) Get rid of *.conf.ext files. Make everything part of dovecot.conf, so doveconf -n outputs ALL of the configuration. There are mainly 3 config files I'm thinking about: dict-sql, passdb/userdb sql, passdb/userdb ldap. The dict-sql is something I think needs a bigger redesign (mentioned above in "Metadata" section), but the sql/ldap auth configs could be merged. One way could be: sql_db sqlmails { # most settings from dovecot-sql.conf.ext, except for queries driver = mysql connect = ... } ldap_db ldapmails { # most settings from dovecot-ldap.conf.ext, except attributes/filters } passdb { driver = sql db = sqlmails sql_query = select password from users where username = '%u' } passdb { driver = ldap db = ldapmails ldap_attributes { password = %{ldap:userPassword} } ldap_filter = ... } The sql_db {} and ldap_db {} would be generic enough to be used everywhere (e.g. dict-sql), not just for passdb/userdb. Some problems: - Similar to the per-namespace mail settings, doveconf -a would output all sql_query, ldap_attributes, ldap_filter, etc. settings for all passdbs/userdbs. Perhaps a similar solution? - The database configs contain passwords, so they should be readable only by root. This makes running dovecot-lda and maybe doveadm difficult, since they fail at "permission denied" when trying to open the config. There are probably only two solutions: a) The db configs need to be !include_try'd or b) the configs can be world-readable, but only passwords are placed to only-root-readable files by using "password = <db.password" IMAP state saving/restoring --------------------------- IMAP connections are often long running. Problems with this: 1. Currently each connection requires a separate process (at least to work reliably), which means each connection also uses quite a lot of memory even when they aren't doing anything for a long time. 2. Some clients don't handle lost connections very nicely. So Dovecot can't be upgraded without causing some user annoyance. Also in a cluster if you want to bring down one server, the connections have to be disconnected before they can be moved to another server. If IMAP session state could be reliably saved and later restored to another process, both of the above problems could be avoided entirely. Typically when a connection is IDLEing there are really just 4 things that need to be remembered: username, selected mailbox name, its UIDVALIDITY and HIGHESTMODSEQ. With this information the IMAP session can be fully restored in another process without losing any state. So, what we could do is: 1. When an IMAP connection has bee IDLEing for a while (configurable initial time, could be dynamically adjusted): - move the IMAP state and the connection fd to imap-idle process - the old imap process is destroyed - imap-idle process can handle lots of IMAP connections - imap-idle process also uses inotify/etc. to watch for changes in the specified mailbox - if any mailbox changes happen or IMAP client sends a command, start up a new imap process, restore the state and continue from where we left off - This could save quite a lot of memory at the expense of some CPU usage 2. Dovecot proxy <-> backend protocol could be improved to support moving connection to another backend. Possibly using a separate control connection to avoid making the proxying less efficient in normal operation. 3. When restarting Dovecot, move all the connections to a process that keeps the connections open for a while. When Dovecot starts up, create imap processes back to the connections. This allows changing configuration for existing client connections (which sometimes may be bad! need to add checks against client-visible config conflicts), upgrading Dovecot, etc. without being visible to clients. The only problem is SSL connections: OpenSSL doens't provide a way to save/restore state, so either you need to set shutdown_clients=no (and possibly keep some imap-login processes doing SSL proxying for a long time), or SSL connections need to be killed. Of course the SSL handling could be outsourced to some other software/hardware outside Dovecot. The IMAP state saving isn't always easy. Initially it could be implemented only for the simple cases (which are a majority) and later extended to cover more. IMAP extensions --------------- - CATENATE is already implemented by Stephan - URLAUTH is also planned to be implemented, somewhat differently than in Apple's patch. The idea is to create a separate imap-urlauth service that provides extra security. - NOTIFY extension could be implemented efficiently using mailbox list indexes, which already exists in v2.1. - FILTERS extension can be easily implemented once METADATA is implemented - There are also other missing extensions, but they're probably less important: BINARY & URLAUTH=BINARY, CONVERT, CONTEXT=SORT, CREATE-SPECIAL-USE, MULTISEARCH, UTF8=* and some i18n stuff. Backups ------- Filesystem based backups have worked well enough with Dovecot in the past. But with new features like single instance storage it's becoming more difficult. There's no 100% consistent way to even get filesystem level backups with SIS enabled, because deleting both the message file and its attachment files can't be done atomically (although usually this isn't a real problem). Restoring SIS mails is more difficult though, first you need to restore the dbox mail files and then you need to figure out what attachment files from SIS need to be restored, and finally you'll need to do doveadm import to put them into their final destination. I don't have much experience with backup software, but other people in my company do. The initial idea is to implement a Dovecot backup agent to one (commercial) backup software, which allows doing online backups and restoring mails one user/mailbox/mail at a time. I don't know the details yet how exactly this is going to be implemented, but the basic plan is probably to implement a "backup" mail storage backend, which is a PostgreSQL pg_dump-like flat file containing mails from all mailboxes. doveadm backup/import can then export/import this format via stdout/stdin. Incremental backups could possibly be done by giving a timestamp of previous backup run (I'm not sure about this yet). Once I've managed to implement the first fully functional backup agent, it should become clearer how to implement it to other backup solutions. Random things ------------- - dovecot.index.cache file writing is too complex, should be simplified - Enable auth_debug[_passwords]=yes on-the-fly for some specific users/IPs via doveadm - Optimize virtual mailboxes using mailbox list indexes. It wouldn't anymore need to keep all the backend mailboxes' index files open. - Would be nice to go forward with supporting key-value databases as mail storage backends.

5 11

[Dovecot] Sieve_extprograms plugin for Pigeonhole v0.3
by Stephan Bosch 16 Feb '12

16 Feb '12

As indicated in the release announcement of Pigeonhole v0.3.0, there is also a new experimental plugin available for Pigeonhole v0.3: the sieve_extprograms plugin. This can be used to pipe messages to and filter messages through external programs (shell scripts). Also, string data can be retrieved from such external programs. The plugin can currently only be obtained from its Mercurial repository: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms Refer to the INSTALL and README files for more information: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/INSTALL http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/README The vendor-specific Sieve language extension provided by this plugin are documented here: http://hg.rename-it.nl/pigeonhole-0.3-sieve-extprograms/raw-file/tip/doc/rf… Please test this plugin. The plan is to include it into the main Pigeonhole release eventually. Regards, Stephan.

1 0

Re: [Dovecot] [Dovecot 2.1] imap service crash when using some clients
by interfaSys sàrl 16 Feb '12

16 Feb '12

Hello Pascal, I've just tested rc6 and get almost the same results. imap still crashes, but Horde still works. I forgot to mention that this is on FreeBSD 8 using gcc 4.6 and LTO. And contrary to what I said in my initial email, it doesn't seem to be ACL plugin related. imap crashes even if it's not in the plugins list. Here is the backtrace: # gdb /usr/libexec/dovecot/imap /path/to/core Core was generated by `imap'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/lib/dovecot/libdovecot-storage.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot-storage.so.0 Reading symbols from /usr/lib/dovecot/libdovecot.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/libdovecot.so.0 Reading symbols from /lib/libc.so.7...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.7 Reading symbols from /usr/lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib/librt.so.1 Reading symbols from /usr/lib/dovecot/lib01_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib01_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib02_imap_acl_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib02_imap_acl_plugin.so Reading symbols from /usr/lib/dovecot/lib10_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib10_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_imap_quota_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_imap_quota_plugin.so Reading symbols from /usr/lib/dovecot/lib11_trash_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib11_trash_plugin.so Reading symbols from /usr/lib/dovecot/lib20_autocreate_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_autocreate_plugin.so Reading symbols from /usr/lib/dovecot/lib20_fts_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_fts_plugin.so Reading symbols from /usr/lib/dovecot/lib20_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib20_zlib_plugin.so Reading symbols from /lib/libz.so.5...(no debugging symbols found)...done. Loaded symbols for /lib/libz.so.5 Reading symbols from /usr/lib/libbz2.so.4...(no debugging symbols found)...done. Loaded symbols for /usr/lib/libbz2.so.4 Reading symbols from /usr/lib/dovecot/lib21_fts_squat_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib21_fts_squat_plugin.so Reading symbols from /usr/lib/dovecot/lib30_imap_zlib_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib30_imap_zlib_plugin.so Reading symbols from /usr/lib/dovecot/lib90_antispam_plugin.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib/dovecot/lib90_antispam_plugin.so Reading symbols from /libexec/ld-elf.so.1...(no debugging symbols found)...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00000000105c64e7 in imap_match () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00000000104b4bf6 in mailbox_list_iter_next () from /usr/lib/dovecot/libdovecot-storage.so.0 No symbol table info available. #2 0x0000000000416b99 in cmd_list_continue.6853 () No symbol table info available. #3 0x000000000042166d in cmd_list_full () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x8080808080808080 in ?? () No symbol table info available. #6 0x0000000000000004 in ?? () No symbol table info available. #7 0x0000000010a3f1a8 in ?? () No symbol table info available. #8 0x0000000010aa4390 in ?? () No symbol table info available. #9 0x0000000010a3f174 in ?? () No symbol table info available. #10 0x00007fffffffe9a0 in ?? () No symbol table info available. #11 0x0000000010596e4f in p_strdup () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #12 0x0000000010a22200 in ?? () No symbol table info available. #13 0x0000000010aa4458 in ?? () No symbol table info available. #14 0x0000000010aa4340 in ?? () No symbol table info available. #15 0x0000000010aa4498 in ?? () No symbol table info available. #16 0x0000000110a3f040 in ?? () No symbol table info available. #17 0x0000000010a3f128 in ?? () No symbol table info available. #18 0x0000000010a3e840 in ?? () No symbol table info available. #19 0x0000000010a3f040 in ?? () No symbol table info available. #20 0x0000000010a22200 in ?? () No symbol table info available. #21 0x0000000000000001 in ?? () No symbol table info available. #22 0x0000000010a22200 in ?? () No symbol table info available. #23 0x0000000000000000 in ?? () No symbol table info available. #24 0x000000000041d320 in client_command_input.12433 () No symbol table info available. #25 0x00000000004203ca in client_handle_input () No symbol table info available. #26 0x0000000010a574e0 in ?? () No symbol table info available. #27 0x00000000105ae219 in i_stream_read () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #28 0x000000000041d13f in client_input () No symbol table info available. #29 0x00000000105ce276 in io_loop_call_io () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #30 0x00000000105c9ea3 in io_loop_handler_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #31 0x00000000105c9f78 in io_loop_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #32 0x00000000105c9fb3 in master_service_run () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #33 0x00000000004158bb in main () No symbol table info available. Regards, Olivier > On 01/18/2012 02:03 AM interfaSys sàrl wrote: > >/ Hello, > />/ > />/ I've just noticed that when Horde is connecting to Dovecot 2.1, it > />/ crashes the imap service if Dovecot is configured to use the ACL plugin. > />/ I'm not sure what's so special about the command Horde sends, but it > />/ shouldn't make Dovecot crash. Everything is fine when using Thunderbird. > />/ > />/ Here is the message in Dovecot's logs > />/ "Fatal: master: service(imap): child 89974 killed with signal 11 (core > />/ not dumped)" > />/ > />/ The message says that the core is not dumped, even though I did add > />/ drop_priv_before_exec=yes to my config file. > / > dovecot stop > ulimit -c unlimited > dovecot > > Now connect with Horde and let it crash. > > >/ I've tried connecting to the pid using gdb, but the process just hangs > />/ as soon as I'm connected. > />/ > / > continue > [wait for the crash] > bt full > detach > quit > > > Regards, > Pascal

2 7

[Dovecot] v2.1.0 released
by Timo Sirainen 16 Feb '12

16 Feb '12

http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz http://dovecot.org/releases/2.1/dovecot-2.1.0.tar.gz.sig Here's finally the the stable v2.1 release. I expect this release to be quite stable, since it's already being used in several large installations. Changes since rc7: FS layout related crashfix, two man pages added. The biggest changes since v2.0: * Plugins now use UTF-8 mailbox names rather than mUTF-7: acl, autocreate, expire, trash, virtual * auth_username_format default changed to %Lu. If you really want case sensitive usernames, set it back to empty. * Solr full text search backend changed to use mailbox GUIDs instead of mailbox names, requiring reindexing everything. solr_old backend can be used with old indexes to avoid reindexing, but it doesn't support some newer features. * Expire plugin: Only go through users listed by userdb iteration. Delete dict rows for nonexistent users, unless expire_keep_nonexistent_users=yes. * Temporary authentication failures sent to IMAP/POP3 clients now includes the server's hostname and timestamp. This makes it easier to find the error message from logs. * dsync was merged into doveadm. There is still "dsync" symlink pointing to "doveadm", which you can use the old way for now. The preferred ways to run dsync are "doveadm sync" (for old "dsync mirror") and "doveadm backup". + imapc (= IMAP client) storage allows using a remote IMAP server to be used as storage. This allows using Dovecot as a smart (caching) proxy or using dsync to do migration from remote IMAP server. + Mailbox indexing via queuing indexer service (required for Lucene) + Lucene full text search (FTS) backend rewritten with support for different languages + FTS finally supports "OR" search operation + FTS supports indexing attachments via external programs + IMAP FUZZY extension, supported by Lucene and Solr FTS backends + IMAP SPECIAL-USE extension to describe mailboxes + Mailbox list indexes + Statistics tracking via stats service. Exported via doveadm stats. + Autocreate plugin creates/subscribes mailboxes physically only when the mailbox is opened for the first time. Mailbox listing shows the autocreated mailboxes even if they don't physically exist. + Password and user databases now support default_fields and override_fields settings to specify template defaults/overrides. + SCRAM-SHA-1 authentication mechanism by Florian Zeitz + LDAP: Allow building passdb/userdb extra fields from multiple LDAP attributes by using %{ldap:attributeName} variables in the template. + Improved multi-instance support: Track automatically which instances are started up and manage the list with doveadm instance commands. All Dovecot commands now support -i <instance_name> parameter to select the instance (instead of having to use -c <config path>). See instance_name setting. + auth: Implemented support for Postfix's "TCP map" sockets for user existence lookups. - listescape plugin works perfectly now

3 7

[Dovecot] There is a way to know if a email has been expunged ?
by Alex Baule 16 Feb '12

16 Feb '12

Hy Everyone...and Timo ! There is a way to know if a email was expunged (deleted from hard disk) inside a plugin ? I rewrite the expunge function, but the real expunge function is void, i can't know if was really expunged. There is a way to know this ?

2 8

[Dovecot] Any possibility of running query after sucessful login?
by Jacek Osiecki 16 Feb '12

16 Feb '12

Hi, I'm using finally dovecot 2.0 after quite a smooth upgrade from 1.0 :) with virtual mailboxes stored in sql database. Sometimes I need to check if a mailbox has been used recently or not. Checking for logins in dovecot logs sometimes means searching through gigabytes of info... I was just wondering if there is any possibility of running another query after successful login - just to fill some extra field like "last_login"? So far my only idea is to replace user/password query with procedure like login(user,password) that would also return a line as required by user/password query. However, some mysql implementations (for example, standard php-mysql) can't handle procedures which return values - so will it work in dovecot? Or maybe some other ideas how to do that? Greetings, -- Jacek Osiecki joshua(a)ceti.pl GG:3828944 I don't want something I need. I want something I want.

2 4

[Dovecot] index synchronization during local delivery
by Andy YB Hu 16 Feb '12

16 Feb '12

Hi there, Now our system will take dovecot as the whole imap back end, and keep the LDA of ours, But our system has our own UID generation mechanism (in dovecot it's incremental from 1), which will be done while local delivery, So when local deliveries, it will first generate the UID, move the mail into maildir, then synchronize the mailbox index, which is part of dovecot LDA does. The question is what API or something else about index synchronization I can integrate into our own LDA program easily? Seems mailbox_copy() would be the possibility? Thanks.

2 7

Re: [Dovecot] 2.1b6 imapc sync error
by Metro Domain Admin 15 Feb '12

15 Feb '12

On Wed, Feb 15, 2012, at 10:50 PM, Timo Sirainen wrote: > On 15.2.2012, at 22.48, Metro Domain Admin wrote: > > > On Wed, Feb 15, 2012, at 09:36 PM, Timo Sirainen wrote: > >> Yes, it's because of "sync". The "Not supported" comes from imapc > >> backend, because Dovecot tried to do some changes to it that weren't > >> possible with IMAP protocol (modseq changes probably). I think the > >> migration still was successful though. > > > > The migration went beautifully otherwise. But are the errors minor > > enough that I can continue with sync? > > Probably better to continue with "backup" just in case. :) > > > I had the idea that I could sync back to the imapc account, as a > > fallback option (and, presumably I'd want to preserve the /tmp/imapc > > directory for this?). > > That's not something that I've tested, and I don't know how well it works > (if at all). > OK, "backup" it is then. Thanks.

1 0

[Dovecot] MySQL server has gone away
by Mark Moseley 15 Feb '12

15 Feb '12

I'm running 2.0.17 and I'm still seeing a decent amount of "MySQL server has gone away" errors, despite having multiple hosts defined in my auth userdb 'connect'. This is Debian Lenny 32-bit and I'm seeing the same thing with 2.0.16 on Debian Squeeze 64-bit. E.g.: Jan 12 20:30:33 auth-worker: Error: mysql: Query failed, retrying: MySQL server has gone away Our mail mysql servers are busy enough that wait_timeout is set to a whopping 30 seconds. On my regular boxes, I see a good deal of these in the logs. I've been doing a lot of mucking with doveadm/dsync (working on maildir->mdbox migration finally, yay!) on test boxes (same dovecot package & version) and when I get this error, despite the log saying it's retrying, it doesn't seem to be. Instead I get: dsync(root): Error: user ...: Auth USER lookup failed dsync(root): Fatal: User lookup failed: Internal error occurred. Refer to server log for more information. Watching tcpdump at the same time, it looks like it's going through some of the mysql servers, but all of them have by now disconnected and are in CLOSE_WAIT. Here's an (edited) example after doing a dsync that completes without errors, with tcpdump running in the background: # sleep 30; netstat -ant | grep 3306; dsync -C^ -u mailbox(a)test.com backup mdbox:~/mdbox tcp 1 0 10.1.15.129:57436 10.1.52.48:3306 CLOSE_WAIT tcp 1 0 10.1.15.129:49917 10.1.52.49:3306 CLOSE_WAIT tcp 1 0 10.1.15.129:35904 10.1.52.47:3306 CLOSE_WAIT 20:49:59.725005 IP 10.1.15.129.35904 > 10.1.52.47.3306: F 1126:1126(0) ack 807 win 1004 <nop,nop,timestamp 312603858 766667259> 20:49:59.725459 IP 10.1.52.47.3306 > 10.1.15.129.35904: . ack 1127 win 123 <nop,nop,timestamp 766667998 312603858> 20:49:59.725568 IP 10.1.15.129.57436 > 10.1.52.48.3306: F 1126:1126(0) ack 807 win 1004 <nop,nop,timestamp 312603858 1842560856> 20:49:59.725779 IP 10.1.52.48.3306 > 10.1.15.129.57436: . ack 1127 win 123 <nop,nop,timestamp 1842561225 312603858> dsync(root): Error: user mailbox(a)test.com: Auth USER lookup failed dsync(root): Fatal: User lookup failed: Internal error occurred. Refer to server log for more information. 10.1.15.129 in this case is the dovecot server, and the 10.1.52.0/24 boxes are mysql servers. That's the same pattern I've seen almost every time. Just a FIN packet to two of the servers (ack'd by the mysql server) and then it fails. Is the retry mechanism supposed to transparently start a new connection, or is this how it works? In connecting remotely to these same servers (which aren't getting production traffic, so I'm the only person connecting to them), I get seemingly random disconnects via IMAP, always coinciding with a "MySQL server has gone away" error in the logs. This is non-production, so I'm happy to turn on whatever debugging would be useful. Here's doveconf -n from the box the tcpdump was on. This box is just configured for lmtp (but have seen the same thing on one configured for IMAP/POP as well), so it's pretty small, config-wise: # 2.0.17: /etc/dovecot/dovecot/dovecot.conf # OS: Linux 3.0.9-nx i686 Debian 5.0.9 auth_cache_negative_ttl = 0 auth_cache_ttl = 0 auth_debug = yes auth_failure_delay = 0 base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot/debug.log default_client_limit = 3005 default_internal_user = doveauth default_process_limit = 1500 deliver_log_format = M=%m, F=%f, S="%s" => %$ disable_plaintext_auth = no first_valid_uid = 199 last_valid_uid = 201 lda_mailbox_autocreate = yes listen = * log_path = /var/log/dovecot/mail.log mail_debug = yes mail_fsync = always mail_location = maildir:~/Maildir:INDEX=/var/cache/dovecot/%2Mu/%2.2Mu/%u mail_nfs_index = yes mail_nfs_storage = yes mail_plugins = zlib quota mail_privileged_group = mail mail_uid = 200 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mdbox_rotate_interval = 1 days mmap_disable = yes namespace { hidden = no inbox = yes list = yes location = prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = /opt/dovecot/etc/lmtp/sql.conf driver = sql } plugin { info_log_path = /var/log/dovecot/dovecot-deliver.log log_path = /var/log/dovecot/dovecot-deliver.log quota = maildir:User quota quota_rule = *:bytes=25M quota_rule2 = INBOX.Trash:bytes=+10%% quota_rule3 = *:messages=3000 sieve = ~/sieve/dovecot.sieve sieve_before = /etc/dovecot/scripts/spam.sieve sieve_dir = ~/sieve/ zlib_save = gz zlib_save_level = 3 } protocols = lmtp sieve service auth-worker { unix_listener auth-worker { mode = 0666 } user = doveauth } service auth { client_limit = 8000 unix_listener login/auth { mode = 0666 } user = doveauth } service lmtp { executable = lmtp -L process_min_avail = 10 unix_listener lmtp { mode = 0666 } } ssl = no userdb { driver = prefetch } userdb { args = /opt/dovecot/etc/lmtp/sql.conf driver = sql } verbose_proctitle = yes protocol lmtp { mail_plugins = zlib quota sieve } Thanks!

4 29

[Dovecot] 2.1b6 imapc sync error
by Metro Domain Admin 15 Feb '12

15 Feb '12

I received this error at the end of a migration from imapc to mdbox. Is it because I wanted doveadm 'sync' instead of 'backup'? I didn't get time to do any testing, but since you're pushing for the release, here it is: kerio2:~ adminuser$ /opt/dovecot/bin/doveadm -o imapc_user=jsample -o imapc_password=****** -o imapc_host=*.*.*.* sync -R -u jsample_dc(a)mydomain.com imapc:/tmp/imapc dsync(jsample_dc(a)mydomain.com): Error: Can't update mailbox Archives/2011: Not supported dsync(jsample_dc(a)mydomain.com): Error: Can't update mailbox Archives/2012: Not supported dsync(jsample_dc(a)mydomain.com): Error: Can't update mailbox INBOX: Not supported ...etc, for all mailboxes in the account. This is my userdb entry: jsample_dc@mydomain.com:{***}password:502:6:IMAPC RECIPIENT::/bin/false: kerio2:~ adminuser$ /opt/dovecot/bin/doveconf -n # 2.1.rc6: /opt/dovecot-2.1.rc6/etc/dovecot/dovecot.conf # OS: Darwin 9.8.0 Power Macintosh auth_cache_negative_ttl = 0 auth_cache_size = 1 M auth_debug = yes auth_mechanisms = plain login auth_username_format = %n(a)mydomain.com auth_verbose = yes disable_plaintext_auth = no instance_name = dovecot-2.1.rc6 listen = 192.168.20.40 mail_home = /var/mail/%d/%n mail_location = mdbox:~/mdbox mail_plugin_dir = /opt/dovecot/lib/dovecot mail_plugins = fts fts_squat # I know, this is deprecated in 2.1 mail_privileged_group = mail maildir_very_dirty_syncs = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = scheme=CRYPT username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } plugin { fts = squat } postmaster_address = postmaster(a)mydomain.com protocols = imap lmtp service auth { unix_listener /var/spool/postfix/private/auth { group = _postfix mode = 0660 user = _postfix } unix_listener auth-userdb { group = mail mode = 0660 user = adminuser } } service imap { executable = imap postlogin } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = _postfix mode = 0666 user = _postfix } } service postlogin { executable = script-login rawlog -t } service stats { fifo_listener stats-mail { mode = 00 } } ssl_ca = </opt/local/apache2/conf/ssl/gd_bundle.crt ssl_cert = </opt/local/apache2/conf/ssl/server4.crt ssl_key = </opt/local/apache2/conf/ssl/server4.key userdb { args = username_format=%u /opt/dovecot/etc/dovecot/users driver = passwd-file } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 16 } kerio2:~ adminuser$

2 1

[Dovecot] domain quota in sql
by Adam Szpakowski 15 Feb '12

15 Feb '12

Hi, I have a problem with setting up quota usage tracking for domains. Problem is, that domain quota is always based on only one user in domain, the last one on whom behalf any operation was performed. To recreate this situation, one has only to perform: doveadm quota recalc -u USER on a user, and the domain quota takes the value of this user quota. Have anyone a clue what could cause this problem? Quota tracking is set up both for users and domains. For users quota tracking is working fine, data is saved into dbase without problems. We are using dovecot 2.0.15. Quota enforcing (rules) is NOT a part of the problem. Important parts of config: dict { quota = mysql:/etc/dovecot/dovecot-dict-used-quota.conf.ext quota2 = mysql:/etc/dovecot/dovecot-dict-used-quota-domain.conf.ext } plugin { quota = dict:User quota::proxy::quota quota2 = dict:Domain quota:%d:proxy::quota2 } /etc/dovecot/dovecot-dict-used-quota.conf.ext map { pattern = priv/quota/storage table = used_quota username_field = username value_field = bytes } map { pattern = priv/quota/messages table = used_quota username_field = username value_field = messages } /etc/dovecot/dovecot-dict-used-quota-domain.conf.ext map { pattern = priv/quota/storage table = used_quota_domain username_field = domain value_field = bytes } map { pattern = priv/quota/messages table = used_quota_domain username_field = domain value_field = messages } -- Adam Szpakowski

4 10

[Dovecot] Dovecot-2.0.18 and Dovecot-Pigeonwhole-0.2.6
by Ralf Zimmermann 15 Feb '12

15 Feb '12

Hi all, today I have compiled Dovecot-2.0.18 with Dovecot-Pigeonwhole-0.2.6. Now I have following Problem with sieve: ------------------------%<--------------------------------------------- main_script: line 249: error: included personal script 'vacation.sieve' does not exist. main_script: error: validation failed. ------------------------%<--------------------------------------------- I have a definition for 'sieve_dir' and the sieve script 'vacation.sieve' exits in the folder. # Dovecot Sieve Parameters plugin { ... sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } If I use Dovecot-2.0.17 and Dovecot-Pigeonwhole-0.2.5 with the same configuration I can define include :personal "vacation.sieve"; in my main sieve script. All works fine. Greets Ralf

3 3

Re: [Dovecot] 2.1.rc5 Cannot delete folders from trash
by Daniel L. Miller 15 Feb '12

15 Feb '12

On 2/14/2012 4:42 PM, Willie Gillespie wrote: > On 02/14/2012 01:40 PM, Daniel L. Miller wrote: >> On 2/14/2012 12:06 PM, Willie Gillespie wrote: >>> On 2/13/2012 8:34 PM, Daniel L. Miller wrote: >>>> I'm fighting the same issue. With the current release (2.1rc6) no >>>> errors >>>> are logged - but I'm having problems deleted folders from trash. I've >>>> enabled tb-extra-mailbox-sep. Two folders that were previously in >>>> Trash >>>> were successfully removed. I deleted four more - after several >>>> Thunderbird restarts, compacts, refreshes, I have killed three of >>>> them - >>>> there is still one folder that won't "die". Thunderbird reports the >>>> mailbox doesn't exist, yet it continues to appear. >>> >>> Out of curiosity -- does the mailbox actually exist if you look on the >>> Dovecot server? >>> Secondly, is it listed in your subscription file? >> >> At one time - yes they were. After further playing, they do not exist in >> the on-disk folder structure, nor in the subscription file. But still >> show in the client - with errors. > > Sounds like a Thunderbird bug then. With Thunderbird shut down, you > could browse into the profile. > ~\.thunderbird or > C:\Users\name\AppData\Roaming\Thunderbird\whatever\ImapMail\server.name > > Find the mailbox that is not supposed to be there and move/delete it. > The filename will usually just be the mailbox name. A .msf file is > Thunderbird's index file. It will create .sbd folders for any > sub-mailboxes. I tried going in via squirrelmail - saw no subfolders under trash. Went back in with Thunderbird - all gone. -- Daniel

1 0

[Dovecot] pop3 not autocreating directory structure
by Jan-Frode Myklebust 15 Feb '12

15 Feb '12

We use: mail_home = /srv/mailstore/%256LRHu/%Ld/%Ln mail: mdbox:~/mdbox and I just noticed one of our newly provisioned users initially failed to pop her mails. I saw several of these: dovecot:: pop3(new.user(a)example.net): Error: Couldn't open INBOX: Mailbox doesn't exist: INBOX dovecot:: pop3(new.user(a)example.net): Couldn't open INBOX top=0/0, retr=0/0, del=0/0, size=0 before she switched to imap and then everything looked fine: dovecot:: imap(new.user(a)example.net): Disconnected: Logged out bytes=11/338 Looking at the timestamps in the filesystem I see that the users home directory wasn't created before switcing to imap. Is this a know problem? -jf

2 3

[Dovecot] v2.1.rc7 released
by Timo Sirainen 15 Feb '12

15 Feb '12

http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz http://dovecot.org/releases/2.1/rc/dovecot-2.1.rc7.tar.gz.sig Okay, one more RC, because I needed a new feature implemented and forgot I wasn't supposed to do any changes anymore before v2.1.0. After that I figured I'll just do a few more changes and release RC7. So, v2.1.0 will probably be released within 1-2 days. So, the one new feature is: + Added ignore_on_failure setting for namespaces. If namespace initialization fails with this enabled (e.g. permission denied), the namespace is silently skipped for the user. Also (not really worth mentioning in NEWS) if you now connect to imap/pop3 port when auth process is misconfigured and dies, instead of hanging the client for 5 mins it gets disconnected pretty much immediately.

1 0

[Dovecot] R: Re: LMTP : Can't handle mixed proxy/non-proxy destinations
by cinzia.delcoiro＠libero.it 15 Feb '12

15 Feb '12

Hi Timo, I'm writing to alert you that even in version 2.1. we have the error: "Can't handle mixed proxy/non-proxy destinations" when we write to two users: one existent and one not. The error occurs randomly. I configure LDAP query in the file : dovecot-ldap.conf in this way: pass_attrs = mail=user,userPassword=password,=proxy=y,mailHost=host, =proxy_timeout=600 specifying "=proxy=y" in order to proxy every mail. As workaround, can I configure a static "host" for non-existing users? This is my dovecot -n # 2.1.rc5: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-194.el5 x86_64 Red Hat Enterprise Linux Server release 5.5 (Tikanga) auth_mechanisms = plain login base_dir = /usr/local/var/run/dovecot/ disable_plaintext_auth = no listen = 127.0.0.1 lmtp_proxy = yes mail_gid = mailusers mail_location = maildir:/mnt/mbox1/mailusers/%n/Maildir mail_uid = mailusers passdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { quota = maildir:User Quota } protocols = lmtp service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 vsz_limit = 256 M } service lmtp { inet_listener lmtp { port = 24 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = no syslog_facility = local5 userdb { args = /usr/local/etc/dovecot/dovecot-ldap.conf driver = ldap } protocol imap { mail_plugins = quota imap_quota } protocol lmtp { postmaster_address = info(a)amm2.miur.it } Regards Cinzia >----Messaggio originale---- >Da: tss(a)iki.fi >Data: 28-gen-2012 18.32 >A: "Alexis Lelion"<alexis.lelion(a)gmail.com> >Cc: "Dovecot Mailing List"<dovecot(a)dovecot.org> >Ogg: Re: [Dovecot] LMTP : Can't handle mixed proxy/non-proxy destinations > >On 27.1.2012, at 12.59, Alexis Lelion wrote: > >> Jan 25 09:05:12 mail01 postfix/lmtp[23934]: A92709300DB: to=< >> user_on_mail02(a)domain.com>, relay=mail01.domain.com[private/dovecot-lmtp], >> delay=0.07, delays=0.01/0/0/0.06, dsn=4.3.0, status=deferred (host >> mail01.domain.com[private/dovecot-lmtp] said: 451 4.3.0 < >> user_on_mail02(a)domain.com> Can't handle mixed proxy/non-proxy destinations >> (in reply to RCPT TO command)) >> >> I was wondering if there was another way of handling this, for example >> by triggering an immediate queue lookup from postfix or forwarding a >> copy of the mail to the other server. Note that the postfix >> "queue_run_delay" was increased to 15min on purpose, so I cannot change >> that. > >It would be possible to change the code to support mixed destinations, but it's probably not a simple change and I have other things to do.. > >Maybe you could work around it so that LMTP always proxies the mails, to localhost as well, but to a different port which doesn't do proxying at all. > >

3 2