- dovecot - dovecot.org

[Dovecot] Question about IMAP
by Charles Marcus 17 May '09

17 May '09

Hi Timo, I have a question that you can maybe answer off the top of your head, considering your vast knowledge of the IMAP protocol... There is a Thunderbird bug that is near and dear to my heart, that is trying to address how Thunderbird deals with large messages (ie, messages with large attachments). Current behavior sucks... it downloads the entire message every time the message is clicked on. My question is: Using the IMAP protocol, is there a way, *without* downloading the entire message (specifically, binary attachments), that an IMAP client can do the following: 1. Download *only* the message headers and the text/html/body contents of the body of the message but *not* any binary attachments, 2. Detect that a message *has* one or more binary attachments, *without* downloading them, 3. Grab the names of all of the attachments, 4. Display for the user the email/email headers and text/html body parts, 5. Display the existence of the attachments in whatever manner the client normally shows them (in TBird, they show at the bottom of the message just above the status bar), 6. Download the actual attachment on demand - ie, if/when it is dbl-click > opened, right-click > saved, or the message is forwarded? Pointers to appropriate IMAP protocol docs would be happily passed on to them the Mozilla dev guys. Thanks! -- Best regards, Charles

2 5

[Dovecot] LDAP auth_bind fails to process certain LDAP server responses (Windows AD)
by Konstantin Khomoutov 15 May '09

15 May '09

LDAP auth backend in Dovecot 1.1.5 seemingly fails to handle certain paths of interaction with LDAP server, which is Windows AD running Windows 2003 server machine in my case. The symptoms I observe look exactly as were already reported in [1]: authentication backend "hangs" after logging the fact it has sent the search request, then after a while the IMAP/POP3 connection is teared down by the server due to idle condition. In the mentioned mail thread, it was proposed that if Dovecot doesn't log any LDAP server response, there is no such response, so I dumped the TCP traffic going between the backend and the server, and studied what happens with different settings. The result is that it seems LDAP auth backend doesn't understand the case when the server offerrs one or more "referrals" which the client might want to check. It seems that the backend tries to do something else after it gets such a result from the LDAP server even if the server returned an entry matching the search request. The described behaviour manifests itself with Windows AD only if I specify the domain name alone as the search base, e.g. "dc=mydomain,dc=com". If I also include at least one OU, the server doesn't return its list of referrals and Dovecot LDAP auth backend works OK. I attached two files produced by tcpdump -- one with "hung" search request, and another with successful one. They are: 1) ldap-hung.pcap.gz Captured with /etc/dovecot/dovecot-ldap.conf settings: hosts = domain007.com dn = saslauthd(a)domain007.com dnpass = "secret" auth_bind = yes base = dc=domain007, dc=com pass_filter = (&(objectClass=person)(sAMAccountName=%u)) Packet #8 is of interest in this dump. 2) ldap-ok.pcap.gz Captured with /etc/dovecot/dovecot-ldap.conf settings: hosts = domain007.com dn = saslauthd(a)domain007.com dnpass = "secret" auth_bind = yes base = cn=Users, dc=domain007, dc=com pass_filter = (&(objectClass=person)(sAMAccountName=%u)) The only difference compared to the first case is the "cn=Users" prepended to the base DN, which prevents the server from attaching a list of referrals. In both cases "search result done" is present in the server response. 3) ldap-ldapsearch.pcap.gz This dump captures the conversation of the ldapsearch utility with the same server using the same parameters Dovecot LDAP auth backend uses in the first case. ldapsearch invocation was like this: $ ldapsearch -h domain007.com -x -D saslauthd(a)domain007.com -w secret -a always -b dc=domain007,dc=com '(&(objectClass=person)(sAMAccountName=kostix))' General configuration info (package from Debian stable): # dovecot --version 1.0.15 # dovecot -n # 1.0.15: /etc/dovecot/dovecot.conf log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login mail_privileged_group: mail mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 pop3_uidl_format(default): pop3_uidl_format(imap): pop3_uidl_format(pop3): %08Xu%08Xv auth default: verbose: yes debug: yes passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: passwd 1. http://www.mail-archive.com/dovecot@dovecot.org/msg09174.html

1 0

[Dovecot] Possibly dumb questions about DC and user/system limits
by Stewart Dean 15 May '09

15 May '09

Because of user access growth, the number of processes associated with IMAP has increased and I thought to change some of the configured parms. dovecot -n yields: # 1.1.14: /usr/local/etc/dovecot.conf # OS: AIX 1 005A928C4C00 listen: *:143 ssl_listen: *:993 disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable: /usr/local/libexec/dovecot/imap-login login_processes_count: 12 login_max_processes_count: 774 max_mail_processes: 1280 mail_max_userip_connections: 12 verbose_proctitle: yes first_valid_uid: 200 mail_location: mbox:~/mail:INBOX=/var/spool/mail/%u:INDEX=/var/dcindx/%u mbox_write_locks: fcntl mbox_dirty_syncs: no auth default: passdb: driver: pam userdb: driver: passwd The only changes was that max_mail_processes went from 1024 to 1280. Now I get a error message when I start DC: Warning: fd limit 2000 is lower than what Dovecot can use under full load (more than 2054). Either grow the limit or change login_max_processes_count and max_mail_processes settings So I changed the no_size and no_size_hard to 3500 and 4000 respectively in both dovecot and root AIX defines nosize:* *Sets the soft limit for the number of file descriptors a user process may have open at one time. an lsuser dovecot returns: dovecot id=417 pgrp=dovecot groups=dovecot shell=/bin/false daemon=true admin=false ... fsize=2097151 cpu=-1 data=262144 stack=65536 core=2097151 rss=65536 nofiles=3500 nofiles_hard=4000 I kill dovecot and all children and restart it....same error message What am I missing? Does the machine have to be rebooted for the no_size to be updated? -- ==== Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, Bard College, New York 12504 sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035

3 4

[Dovecot] cmd_append_continue_parsing assertion failure
by Mike Abbott 15 May '09

15 May '09

Dovecot-1.1.14 on Mac OS X. Panic: IMAP(user): file cmd-append.c: line 266 (cmd_append_continue_parsing): assertion failed: (ctx->count == uid2 - uid1 + 1) Trivially reproducible: $ telnet mailserver 143 a login user password b append inbox Backtrace: 0 libSystem.B.dylib 0x00007fff8458aa92 __kill + 10 1 libSystem.B.dylib 0x00007fff84606a1c abort + 83 2 imap 0x0000000100069f3e default_fatal_finish + 70 3 imap 0x0000000100069f8c i_syslog_fatal_handler + 0 4 imap 0x00000001000696eb i_info + 0 5 imap 0x0000000100001406 cmd_append_continue_parsing + 434 6 imap 0x00000001000017a3 cmd_append + 407 7 imap 0x000000010000677e client_command_input + 33 8 imap 0x000000010000693f client_command_input + 482 9 imap 0x0000000100006a3e client_handle_input + 239 10 imap 0x0000000100006bc2 client_input + 148 11 imap 0x0000000100070f83 io_loop_handler_run + 272 12 imap 0x0000000100070348 io_loop_run + 55 13 imap 0x000000010000db95 main + 1635 14 imap 0x0000000100001010 start + 52

2 1

[Dovecot] Filesystems (was Re: Migration questions...)
by Seth Mattinen 15 May '09

15 May '09

Richard Hobbs wrote: > > Trouble is... i've been googling this as well, just now, and loads of > people say XFS has the better performance, but loads of other people say > ReiserFS has the better performance. It starts to become a religious argument at some point. > We have battery backed up RAID controllers too, in this new system, and > the systems are UPSd, so on that basis i'm still none the wiser! lol > > I appreciate your experience with XFS is a positive one, but even the > dovecot web site says XFS might now be a good choice... > > http://wiki.dovecot.org/MailboxFormat/Maildir > > What a tough decision! I know it probably won't make too much difference > in my situation, but i want this to be a very long-term solution, so > want to do it right first time! > > Any other opinions on XFS vs Reiserfs with Dovecot maildir? > > Thanks again! > > Richard. > If you have time to play "tweak the filesystem": http://everything2.com/index.pl?node_id=1479435 It's a good comparison of etx3, xfs, and reiser from a performance standpoint. Also try this if you want to pull the plug or randomly reboot and see what happens: http://brad.livejournal.com/2116715.html ~Seth

1 0

[Dovecot] Filesystem experience question was Migration questions
by Stewart Dean 15 May '09

15 May '09

Doing a cursory Google scan on journaled Linux filesystems, it seems that the three ground-up journaled FSes: XFS, reiser and JFS all have their separate strong points but all compare favorably. Reiser does a better job with many small files...which would seem to be the reality of maildir formatted inboxes. Any comments on that? Any war stories, that is, any comments on reliability, availability and recoverability of them. Support issues? Possible down the road orphaning? Is FS quotaing support and does it work well with a lot (for me that means 4000) users? Anybody using IBM's open source JFS? I currently use AIX and it's been as dependable as a hammer there. In some 20 years with AIX, I can count on the fingers of one hand the number of times that I've had to use fsck. We had one set of machines that would drop hard 2-3x a week due to a bad main electrical trunk splice....2 or 3 years of that and the worst I ever saw was having to BRS once every two months or so. OTOH, I distrust IBM's commitment to the little shop and affordable entry machines: this is fast vanishing in the pSeries/AIX world...or else we wouldn't be moving towards Linux. Thanks! Feel free to contact me offline if you deem this marginally related to DC. -- ==== Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, Bard College, New York 12504 sdean(a)bard.edu voice: 845-758-7475, fax: 845-758-7035

2 1

[Dovecot] Migration questions...
by Richard Hobbs 15 May '09

15 May '09

Hi All, We are soon to migrate our mail server from one piece of hardware to another and we would like to take this opportunity to optimize things. As a result, we would like to replace "uw-imapd" and "qpopper" with "dovecot". The version we will be installing is 1.1.13-2, as this is what is available through the latest Debian stable backports. We will also be using exim to deliver mail (through dovecot's deliver mechanism, of course). So... We are currently using the mbox format with uw-imapd, and would like to migrate to the fastest solution possible with dovecot on the new hardware. My understanding is that "multi-dbox" is not an option in this version anyway, maildir is OK, but not great, and "single-dbox" is therefore going to be the highest performing solution. Is single-dbox the fastest way to go, considering we're going to be using email in the following ways: - IMAP connections with all email in the Inbox (Gmail-style). - IMAP connections with email split into many IMAP folders. - POP3 connections with no email left on the server. - POP3 connections with *all* email left on the server. All connections check for new mail every 5 minutes (on average) and there are 50-60 users). Also, we are not able to change user behaviour in this instance, unfortunately. Can anyone see any problems with the above proposal? Hopefully not... One problem that may arise is the fact that when we migrate, all msg UIDs will be lost. If i'm not mistaken, this means that all emails will be treated by the mail client as brand new, and if through IMAP, will all go bold, and if through POP3, will all be downloaded again (if still on the server) and therefore duplicated in the mail client. If this is the case, is there anything we can do to stop this happening? Does the "Convert" plugin does this job well? Finally, I have a rough draft of our migration plan - is there anything horribly wrong in it that's going to cause lots of problems, that anyone can spot by any chance? 1. Install Debian with exim, mysqld (for Horde/IMP) and mailman. 2. Run an online update. 3. Rsync homedirs and inboxes onto new server, ready for initial exim configuration. 4. Configure exim as per existing mail server and test that mailing lists and normal email works. You should now have the existing mail delivery solution on the brand new hardware. 5. Once mail delivery is sorted, add "deb http://www.backports.org/debian lenny-backports main contrib non-free" into "/etc/apt/sources.list" and run "aptitude update && aptitude install debian-backports-keyring && aptitude update". 6. Install dovecot (at the time of writing, this was version 1.1.13-2) and configure to use existing mbox files (inboxes in /var/spool/mail/ and IMAP folders in /home/user/mail/) 7. Setup exim to use dovecot's "deliver" mechanism for interacting with the inboxes (which are still in mbox format). 8. Configure the "convert" plugin to begin converting the mail to dbox format. 9. Run something manually (if possible) to convert mailboxes before people connect, so the task is already done by the time the outage is over. 10. Give staff access to new speedy mail server! Thanks in advance, people - any help is greatly appreciated! :-) Richard. -- Richard Hobbs (IT Specialist) Toshiba Research Europe Ltd. - Cambridge Research Laboratory Email: richard.hobbs(a)crl.toshiba.co.uk Web: http://www.toshiba-europe.com/research/ Tel: +44 1223 436999 Mobile: +44 7811 803377

14 37

[Dovecot] IMAP subfolders and MBOX format
by Frank Bonnet 14 May '09

14 May '09

Hello folks I have a problem related to the mbox format I think First we use MBOX format with Dovecot 1.14 A user's here need to create some subfolders in his Imap space. Due to the use of MBOX format this is impossible from thunderbird as you cannot creat a directory in a file :-) So to help him I created manually on the mail server some directories into his imap space, Doing so we are able to see the folder and subfolders but it is not possible to move some other folders into the subfolder via the thunderdird client. Anyone to help me a bit ? Note : on another machine which use the Maildir format I can do whatever I want without problem. Thanks a lot.

2 1

[Dovecot] id:N13388 Disable storage of messages on server when read (pop3)
by Dimitrios Karapiperis 14 May '09

14 May '09

Hi there Is there any way in dovecot to disable the storage of messages on server when they are read/downloaded by the users (POP3). The server keeps the e-mail when users tick the option on their MUA to keep (leave) messages on server. dovecot 1.0.7 POP3 / maildir Thanks in advance Dimitrios Karapiperis postmaster [at] thessaloniki [dot] gr

3 3

[Dovecot] What's faster for getting mails into Rails: Dbmail & direct MySQL or net/pop with maildir
by Olaf Spaarmann 14 May '09

14 May '09

Hi, it's my first message in this list, so please excuse any mistakes! I have a quite specific problem and nobody in the Ruby on Rails lists could help me, because it is actually more a question for mail- experts. So I hope you can help me. I'm planning to set up a mailserver using dovecot. Users of my Rails App can forward their emails (john(a)doe.com) to an personal address (john.doe(a)myapp.com ). These emails will be received by my mailserver. So far so good, the tricky part is to get the mails into my Rails App for processing them. After that they will be deleted (so I'll never accumulate a big ammount of mails on the mailserver). I thought of 2 different approaches. 1) Just use a normal Dovecot setup, using maildir and get the mails with a mailpoller script, connecting to the server via Pop3 2) Use Dbmail for storing the emails, don't use Pop3 but let the mailpoller script connect to the database directly I understand that Dbmail seems to be slower than maildir (in terms of processing many incoming mails) but I'm not sure if a connection via net/pop (of the mailpoller script) is maybe that much slower than mysql that it wouldn't matter at all. What's your opinion? I'm gratefull for every hint. Thanks

2 1