- dovecot - dovecot.org

[Dovecot] Performance problems while running doveadm purge
by Jost Krieger 08 Dec '12

08 Dec '12

I have a rather large and active mdbox (28 GB, 3M mess, 1200 deliveries/day). I usually have no problems working with those mails, and there is some batch processing going on (via doveadm). Every few weeks I try my luck running doveadm purge, and this a) crunches about 5GB (to be expected), b) takes rather long (ok), c) leads to long stretches of blocked mdbox, which is the problem. I always turn off the batch processing, but deliveries take so long that they sometimes give up after timeout (they will come back later), and the IMAP clients get blocked for long minutes. Sometimes even the purge run aborts because of 3-minute-timeouts. What could be going on here (2.1.10)? Jost -- | Jost.Krieger+sig(a)ruhr-uni-bochum.de Please help stamp out spam! | | Postmaster, JAPH, resident answer machine at RUB Comp. Center | | Sincere words are not sweet, sweet words are not sincere. | | Lao Tse, Tao Te King 81 |

3 2

[Dovecot] Dovecot director doveadm with switch "-A" error
by Ramon Frontera 08 Dec '12

08 Dec '12

Hi, we have a problem with our director proxy configuration. When we run on proxy server the doveadm command with -A switch, fails with the error: # doveadm -D quota get -A doveadm(user1): Debug: auth input: user=user1 proxy starttls=any-cert doveadm(user1): Error: Proxy is missing destination host doveadm: Error: Failed to iterate through some users If we execute the command with a single user, director assigns host and goes well, Example: # doveadm -D quota get -u user1 doveadm(user1): Debug: auth input: user=user1 proxy starttls=any-cert host=xx.xx.xx.xx proxy_refresh=450 Quota name Type Value Limit % User quota STORAGE 159401 2621440 6 User quota MESSAGE 341 - 0 We have tried different configurations and all goes wrong with this "-A" switch, Timo please, could you tell us if we have a missconfiguration or its a possible director bug ?? We really want execute the expire plugin on ours directors and them redirect to the backends servers of every user. thanks in advance. Our doveconf -n: # dovecot -n # 2.1.10: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.14.1.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_cache_size = 3 k auth_cache_ttl = 15 mins auth_debug = yes auth_verbose = yes base_dir = /var/run/dovecot/ default_client_limit = 30000 default_process_limit = 5000 dict { expire = mysql:/etc/dovecot/dovecot-dict-expire.conf.ext } director_doveadm_port = 990 director_mail_servers = mailserver1 mailserver2 director_servers = proxy1 doveadm_proxy_port = 24245 lmtp_proxy = yes mail_debug = yes mail_plugins = expire quota managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/dovecot-ldap-users.conf.ext driver = ldap } plugin { expire = Trash expire2 = Trash/* expire_dict = proxy::expire } postmaster_address = postmaster<at>company protocols = imap pop3 lmtp sieve service auth { client_limit = 27048 unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { group = dovecot mode = 0660 } } service director { fifo_listener login/proxy-notify { mode = 0600 user = $default_login_user } inet_listener { port = 9090 } unix_listener director-userdb { mode = 0660 } unix_listener login/director { mode = 0666 } } service doveadm { executable = doveadm-server director inet_listener doveadm-server { port = 24245 } } service imap-login { executable = imap-login director inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service ipc { unix_listener ipc { user = dovecot } } service lmtp { client_limit = 1 inet_listener lmtp { port = 30025 } } service managesieve-login { executable = managesieve-login director } service pop3-login { executable = pop3-login director inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } ssl = required ssl_cert = </etc/pki/tls/certs/imap.crt ssl_key = </etc/pki/tls/private/imap.uib.es_privatekey.pem syslog_facility = local1 userdb { driver = prefetch } userdb { args = /etc/dovecot/dovecot-ldap-users.conf.ext driver = ldap } verbose_proctitle = yes protocol lmtp { auth_socket_path = director-userdb mail_plugins = expire quota sieve quota passdb { args = /etc/dovecot/dovecot-ldap-pass.conf.lmtp driver = ldap } } protocol doveadm { auth_socket_path = director-userdb } protocol imap { mail_max_userip_connections = 20 mail_plugins = expire quota } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } local ip/24/24 { doveadm_password = supersecretpassword } -- ------------------------------------------ Ramon Frontera Gallardo Universitat de les Illes Balears

2 9

[Dovecot] Dovecot G+ community
by Patrick Ben Koetter 08 Dec '12

08 Dec '12

For those who own a G+ account and don't mind seeing colors in a browser once in a while instead of staring at a black/white terminal all day long, join the Dovecot G+ community <https://plus.google.com/communities/112967413753095884716> p@rick -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Joerg Heidrich

1 0

Re: [Dovecot] imap-login - where does the user get filled in from?
by Ben Morrow 07 Dec '12

07 Dec '12

At 3PM -0800 on 6/12/12 Jeff Lacki wrote: > Ben Morrow <ben(a)morrow.me.uk> wrote: > > > > imap-login: Disconnected (no auth attempts in 60 secs): user=<>, > > > rip=192.168.1.1, lip=192.168.1.22, TLS handshaking: Disconnected > > > > Just to be sure: you have replaced the self-signed certificate Dovecot > > installs with one which is signed by a public CA? > > I created my own self-signed certificate with mkcert.sh that comes with > dovecot. Changed my hostname to mail.myhostname.com and the other > stuff, installed etc. An iPhone won't accept a self-signed certificate unless you install it in the system trusted certificate list. I don't know how you do that, or even if it's possible... Ben

3 3

[Dovecot] Cannot STORE \Seen flag on some mails
by Frerich Raabe 07 Dec '12

07 Dec '12

Hi, I'm running Dovecot 1.2.17 on FreeBSD (exact output of 'dovecot -n' is atttached to this mail). The machine is serving a public mailinglist archive which is read-only; all mail arriving for the archive is marked as \Seen using Sieve script. This setup works well most of the time, but I noticed that for *some* mails, the \Seen flag doesn't seem to be stored. Right now I have 31255 mails in one of my folders and I can't seem to mark five of them as \Seen - the others work just fine. I first suspected a client issue so I did a little IMAP session by hand: * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE AUTH=PLAIN AUTH=LOGIN] Dovecot ready. . LOGIN "xxxxxx" "yyyyyy" . OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS ACL RIGHTS=texk] Logged in . SELECT "Lists/Archive/squish" * FLAGS (\Answered \Flagged \Deleted \Seen \Draft $NotJunk) * OK [PERMANENTFLAGS ()] Read-only mailbox. * 31250 EXISTS * 0 RECENT * OK [UNSEEN 27126] First unseen. * OK [UIDVALIDITY 1350573750] UIDs valid * OK [UIDNEXT 31265] Predicted next UID * OK [HIGHESTMODSEQ 9512] Highest . OK [READ-ONLY] Select completed. . SEARCH UNSEEN * SEARCH 27126 27127 28484 29835 29838 . OK Search completed (0.000 secs). . STORE 27126 FLAGS \SEEN . OK Store completed. . SEARCH UNSEEN * SEARCH 27126 27127 28484 29835 29838 . OK Search completed (0.000 secs). . LOGOUT * BYE Logging out . OK Logout completed. Note how the first 'SEARCH UNSEEN' command shows that '27126' is unseen, the subsequent 'STORE' command succeeds - but then 'SEARCH UNSEEN' still shows 27126 as unseen! I have all four logging levels being piped to /var/log/maillog (I verified this by running dovecot --log-error) but the file does not show any problems. I checked the file permissions of the Maildir directories, and it all looks dandy to me. Does anybody have some suggestions how to debug this further, or what the reason for this may be? -- Frerich Raabe - raabe(a)froglogic.com www.froglogic.com - Multi-Platform GUI Testing

2 4

[Dovecot] dsync: error with hierarchy separator
by Jakub Jedelsky 07 Dec '12

07 Dec '12

Hi list! I have a problem with transfering mails from mbox to maildir format. On dovecot wiki I found a dsync tool, which looks good, but returns me an error: # dsync -u mail(a)example.com mirror mbox:~/mail:INBOX=/var/mail/virtuser_808 dsync(root): Fatal: Mail locations must use the same virtual mailbox hierarchy separator (specify separator for the default namespace) I found a solution with namespaces, but it doesn't work for me, there is still same error - I tried to add "-o separator=/" as option or add it to config file as mentioned there [1]. Can you push me to any solution? My dovecot -n: # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-279.5.2.el6.x86_64 x86_64 CentOS release 6.3 (Final) auth_mechanisms = plain login digest-md5 cram-md5 disable_plaintext_auth = no mail_location = maildir:~/.maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mbox_write_locks = fcntl passdb { args = scheme=CRYPT username_format=%Lu /etc/dovecot/dovecot.passwd driver = passwd-file } plugin { autocreate = Spam autosubscribe = Spam sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { args = username_format=%Lu /etc/dovecot/dovecot.passwd driver = passwd-file } protocol imap { mail_plugins = autocreate } [1] http://www.dovecot.org/list/dovecot/2010-September/053186.html Thanks a lot. -jj

1 1

[Dovecot] imap-login - where does the user get filled in from?
by jeep＠rahul.net 07 Dec '12

07 Dec '12

Sorry to ask such a basic question, but I finally got some major hurdles working for imaps on port 993 and Im getting the following when I try to connect from my iphone: imap-login: Disconnected (no auth attempts in 60 secs): user=<>, rip=192.168.1.1, lip=192.168.1.22, TLS handshaking: Disconnected Im using SQL and SSL over 993. I cant seem to find where my user comes from that its blank? Your help is appreciated. Thanks /mf/home/jeep/shell/.signature

3 5

[Dovecot] Message Flag definitions
by Chris Dutton 07 Dec '12

07 Dec '12

Hello, I am in charge of migrating our mail system from dovecot to another server, and I am having to write a script to do this as tools like imapsync are too slow. What I am trying to find is the definitions of the flags that dovecot adds to the Maildir file name it uses for the IMAP messages. For example what the a and b mean in the following: 1176236036.P7260Q1438.sucia:2,RSab I am looking for all definitions not just a and b. Thanks Chris chris(a)rosepointnav.com

2 1

[Dovecot] How can I force FTS reindexing?
by Dave Abrahams 06 Dec '12

06 Dec '12

I threw out my lucene-indexes folder because it seemed to be missing something, and I figured it would get regenerated. Unfortunately, dovecot is only FTS-indexing new mails that come in. Is there a way to get it to reindex everything? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

1 1

Re: [Dovecot] dovecot Digest, Vol 116, Issue 11
by Erik A Johnson 06 Dec '12

06 Dec '12

On December 5, 2012 2:07:14 AM PST, Ben Morrow <ben(a)morrow.me.uk> wrote: > At 1AM -0800 on 5/12/12 Erik A Johnson wrote: >> FYI, the tcpdump I sent previously was with one of our >> previously-discussed patches in place: >> >> if (!proxy->client_proxy && net_geterror(proxy->fd_ssl) == EBADF) { >> >> I'm attaching that dump again (as >> tcpdump_output_witholdpatch_headeronly.txt), as well as a dump without >> any of the patches (tcpdump_output_withoutpatches_headeronly.txt) in >> case there is any difference. > > Well, they're certainly different. Are you sure the second trace > (withoutpatches) was of a session which went into an infinite loop? > The only thing peculiar about that trace is that the server closes the > connection after receiving the first packet from the client, but it does > so perfectly properly: it ACKs the client's data packet, and does the > FIN-FIN/ACK exchance properly. You will notice there are no [R] packets, > which indicate something odd is happening at the server end. I'm pretty sure, but I've run it again, confirmed that the imap-login process is using 100% of a CPU until I kill it, and have attached the tcpdump. Looks like one packet from SERVER to CLIENT shifted slightly in chronology, but otherwise the same. >>> #ifdef SO_ISDEFUNCT >>> + if (getsockopt(proxy->fd_ssl, SOL_SOCKET, SO_ISDEFUNCT, >>> + (void *)&err, sizeof(err)) == 0 && err) { >>> + errstr = t_strdup_printf( >>> + "%s: socket is defunct", func_name); >>> + break; >>> + } >>> +#endif >> >> Nope, SO_ISDEFUNCT isn't defined. > > Oh, sorry, that needs > > #include <sys/socket.h> > > at the top. If that doesn't work, then which version of the OS are you > building for? AFAICT the DEFUNCT socket flag has been present since at > least 10.5, but the SO_ISDEFUNCT option was only introduced in 10.7. > This is irritating, actually: it means that to properly fix this on all > versions of Mac OS Dovecot would need to include the previous ENOTCONN > code #ifndef SO_ISDEFUNCT. I've got both 10.7 and 10.8 SDKs in Xcode and neither have SO_ISDEFUNCT defined in sys/socket.h (or anywhere else in the usr/include directories) -- there's a SS_DEFUNCT mask defined in sys/socketvar.h -- is that what you're looking for?

2 2

[Dovecot] Update configuration for separate mail directory
by eehmke 06 Dec '12

06 Dec '12

I have a small mail server with virtual users. In my /etc/dovecot/conf.d/10- mail.conf I have this setting: mail_location = maildir:/var/mail/vhosts/%d/%u which is also the home location of each user. Now I learned that this is not the prefered configuration, and in fact some of my sieve subdirectories are mistaken as mail directories. While I understand that I have to create separate mail subdirectories in each user folder, I am not sure where I have to adapt the dovecot configuration files. I could not find a guide, only some scattered hints and of course http://wiki2.dovecot.org/VirtualUsers/Home I guess I have to shut down the server for the reconfiguration, or is there a way to do it on a life system? Eggert -- View this message in context: http://dovecot.2317879.n4.nabble.com/Update-configuration-for-separate-mail… Sent from the Dovecot mailing list archive at Nabble.com.

1 0

[Dovecot] When are search indexes updated?
by David Abrahams 06 Dec '12

06 Dec '12

I noticed that occasionally searching in my huge archive mailbox can be really slow, so I tried doveadm index on it and it seemed to do a lot of work, which seemed strange given, for example, that dovecot-lda says it keeps Dovecot index files up-to-date. Then I thought, "maybe these are different files than the search indices." If so, that's not entirely clear from the docs and Wiki. So, questions: * When are search indexes updated? * Are they updated incrementally? * If not, why not? * If so, why would a mailbox's index drift out-of-date, as mine had? BTW, I'm using the clucene search backend. -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

6 16

[Dovecot] "no longer mounted" warnings
by Dave Abrahams 06 Dec '12

06 Dec '12

Dovecot seems to be warning about every volume it's ever seen in the past. Is this normal? Can I make it stop? --8<---------------cut here---------------start------------->8--- 12/4/12 12:33:38.148 PM dovecot[2658]: master: Warning: /Volumes/fs is no longer mounted. See http://wiki2.dovecot.org/Mountpoints 12/4/12 12:33:38.148 PM dovecot[2658]: master: Warning: /Volumes/dave is no longer mounted. See http://wiki2.dovecot.org/Mountpoints 12/4/12 12:33:38.149 PM dovecot[2658]: master: Warning: /Volumes/Groups is no longer mounted. See http://wiki2.dovecot.org/Mountpoints 12/4/12 12:33:38.149 PM dovecot[2658]: master: Warning: /Volumes/Public is no longer mounted. See http://wiki2.dovecot.org/Mountpoints 12/4/12 12:33:38.149 PM dovecot[2658]: master: Warning: /Volumes/Users is no longer mounted. See http://wiki2.dovecot.org/Mountpoints 12/4/12 12:33:38.149 PM dovecot[2658]: master: Warning: /Volumes/GPGTools is no longer mounted. See http://wiki2.dovecot.org/Mountpoints 12/4/12 12:33:38.149 PM dovecot[2658]: master: Warning: /private/tmp/ccc_dmg_mount.weHZmz is no longer mounted. See http://wiki2.dovecot.org/Mountpoints --8<---------------cut here---------------end--------------->8--- -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

4 4

Re: [Dovecot] imap-login hanging when firewall blocks ssl handshaking
by Erik A Johnson 06 Dec '12

06 Dec '12

On December 4, 2012 at 4:43:53 AM PST, Ben Morrow <ben(a)morrow.me.uk> wrote: > So, it looks to me as though you have a firewall problem. You may be > able to get more information by setting the kern.ipc.sodefunctlog sysctl > to 1: this should make the kernel log to syslog (or wherever the OSX > kernel logs go) when sockets are made DEFUNCT and when reads fail for > that reason. sudo sysctl -w kern.ipc.sodefunctlog=1 gives the following in the log: 12/5/12 9:10:00.000 PM kernel[0]: sosetdefunct[60169]: (target pid 60169 level 0) so 0xffffff803159c738 [2,1] marked as defunct 12/5/12 9:10:00.000 PM kernel[0]: sodefunct[60169]: (target pid 60169 level 0) so 0xffffff803159c738 [2,1] is now defunct [rcv_si 0x0, snd_si 0x0, rcv_fl 0x9400, snd_fl 0x1400] 12/5/12 9:10:00.000 PM kernel[0]: soreceive[60169]: defunct so 0xffffff803159c738 [2,1] (57) The last line is repeated about once every 4 microseconds until I kill it.

1 0

[Dovecot] log_path - expansion possible?
by Robert Blayzor 06 Dec '12

06 Dec '12

Does (or can) the ability to use expansion variables in Dovecot's log_path exist? ie: log_path = /var/log/dovecot/dovecot-%Y%m%d -- Robert Blayzor INOC, LLC rblayzor(a)inoc.net http://www.inoc.net/~rblayzor/

3 5

[Dovecot] dotlock error
by Muhammad Yousuf Khan 05 Dec '12

05 Dec '12

i finally manage to control access on public folder by File system permission. i have 3 test users. 1. tom 2. fmaster 3 . testmail tom and fmaster are a group called "news-own" and testmail user is a readonly one. here is my folder structure ill share dovecot -n output at the end of this email. drwxrwxr-t 2 tom news-own 4.0K Dec 4 19:08 tmp drwxrwxr-t 2 tom news-own 4.0K Dec 4 19:08 new -rwxrwxrwx 1 tom news-own 68 Dec 4 19:08 dovecot-uidlist -rw-rw-r-T 1 fmaster news-own 16K Dec 4 19:10 dovecot.index.cache drwxrwxr-t 2 tom news-own 4.0K Dec 4 19:15 cur -rw-rw-r-T 1 tom news-own 1.1K Dec 4 19:16 dovecot.index.log everything works fine as far as user fmaster and tom is concern as they are member of "news-own" and this group own the mailbox now the problem i face with readonly users. i am getting this error in /var/log/mail.info (against user testmail) file_dotlock_create(/public/.News/dovecot-uidlist) failed: Permission denied (euid=1002(testmail) egid=1002(testmail) missing +w perm: /public/.News at first point i saw this error and i could not subscribe to the folder. and the said log has been generated. now on second stage i change the dovecot-uidlist file permission to "1666" so that testmail user can also write to the file. now problem starts from here, i see two folder one is News and second one is News-1 and i don't know from where the second folder showed up. and when i click on any of the folder my /var/log/mail.info start to show below massage continuously. until i restart the dovecot service. file_dotlock_create(/public/.News/dovecot-uidlist) failed: Permission denied (euid=1002(testmail) egid=1002(testmail) missing +w perm: /public/.News i search google and i saw that people suggested to set stickey bit on which i already did or suggested to stop "dot locking" so i am here to ask is there any way to solve my said problem with out disabling dot locking? or if i stop dot locking what are the consequences? here i my dovcecot -n root@mailsrv:/public/.News# dovecot -n # 1.2.15: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.6 log_timestamp: %Y-%m-%d %H:%M:%S ssl_cert_file: /etc/ssl/certs/postfix.pem ssl_key_file: /etc/ssl/private/postfix.key login_dir: /var/run/dovecot/login login_executable: /usr/lib/dovecot/imap-login mail_privileged_group: mail mail_location: maildir:~/Maildir:INBOX=~/Maildir/:INDEX=~/Maildir/index mail_debug: yes mbox_write_locks: fcntl dotlock namespace: type: private separator: / inbox: yes list: yes subscriptions: yes namespace: type: public separator: / prefix: Shared folders/ location: maildir:/public list: yes auth default: mechanisms: plain login passdb: driver: pam userdb: driver: passwd socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix Thanks in advance, any help or suggestion would be highly appreciated.

2 5

[Dovecot] dovecot backend on director ?
by Thomas Hummel 05 Dec '12

05 Dec '12

Hello Timo, The last time I checked my understanding about director was that : 1. backend/"real" imap servers should not be on the same host (or the same dovecot instance) to avoid proxying loops. Is it still the case with the latest 2.1 stable release ? Is there a plan to be able to run director and real server on the same host, thus requiring only 2 hosts instead of 4 to have load balancing and a redundant director ? 2. director itself was not able or meant to check if one of the real server was down (some script was pointed long time ago in this list). Same questions. 3. you'll have to use dovecot lmtp instead of let's say procmail. Would deliver be still a choice or is lmtp the only deliver compatible choice ? In what case would lmtp be more useful than deliver ? To have some sort of "remote LDA" ? By the way, what gets indexed on deposit on the mailbox since, by definition, the user has not done any request yet ? Thanks. -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Groupe Exploitation et Infrastructure

2 2

Re: [Dovecot] imap-login hanging when firewall blocks ssl handshaking
by Erik A Johnson 05 Dec '12

05 Dec '12

FYI, the tcpdump I sent previously was with one of our previously-discussed patches in place: if (!proxy->client_proxy && net_geterror(proxy->fd_ssl) == EBADF) { I'm attaching that dump again (as tcpdump_output_witholdpatch_headeronly.txt), as well as a dump without any of the patches (tcpdump_output_withoutpatches_headeronly.txt) in case there is any difference. On December 4, 2012 6:18:23 AM PST, Ben Morrow <ben(a)morrow.me.uk> wrote: > At 12PM +0000 on 4/12/12 Ben Morrow wrote: >> Well, it looks to me as though xnu/bsd/kern/uipc_socket.c:soreceive will >> indeed return ENOTCONN for a socket which was once successfully >> connected but has now been disconnected. This happens when the socket is >> in the DEFUNCT state, which is a state that doesn't exist in FreeBSD; >> it's not completely clear but I suspect firewalls may be able to put >> arbitrary sockets into that state. > > Investigating a little further, it should be possible to test for this > situation directly. Assuming I'm correct about what's going on here, > this should be both cleaner and safer than mucking about looking for > ENOTCONN and guessing about what's happening. > > Erik, does this make the problem go away? I left out the > proxy->client_proxy test, since AFAICT this is just as likely to happen > on a client socket. > > Ben > > #ifdef SO_ISDEFUNCT > + if (getsockopt(proxy->fd_ssl, SOL_SOCKET, SO_ISDEFUNCT, > + (void *)&err, sizeof(err)) == 0 && err) { > + errstr = t_strdup_printf( > + "%s: socket is defunct", func_name); > + break; > + } > +#endif Nope, SO_ISDEFUNCT isn't defined.

2 1

[Dovecot] Quotas 2.1clarification
by Nick Edwards 05 Dec '12

05 Dec '12

Timo, I have much suspicion that our quota system is not working correctly since we migrate to 2.1 Currently using 2.1.10 Previously we had for issue of warnings quota_warning = storage=90%% /scripts/dqwarn-90.sh quota_warning2 = storage=75%% /scripts/dqwarn-75.sh worked great, these lines also migrated to new conf upon conversion. But quota conf example shows plugin { #quota_warning = storage=95%% quota-warning 95 %u #quota_warning2 = storage=80%% quota-warning 80 %u } # Example quota-warning service. The unix listener's permissions should be # set in a way that mail processes can connect to it. Below example assumes # that mail processes run as vmail user. If you use mode=0666, all system users # can generate quota warnings to anyone. #service quota-warning { # executable = script /usr/local/bin/quota-warning.sh # user = dovecot # unix_listener quota-warning { # user = vmail # } #} So this seems to indicate /scripts/dqwarn-75.sh should be a service name, which then calls the service quota-warning with that script in it etc etc etc surely this old 4 line method has not been replaced into a 20 line mess? Is this old way still functioning as I suspect it is not, and we need to write up two separate new quota warning services given our warnings are worded differently, or should the old methods still be working as they did in 1.2 and we have an error elsewhere? Niki

2 2

[Dovecot] [OT] MS Exchange Alternative?
by Marc Perkel 05 Dec '12

05 Dec '12

Just wondering if there's an open source Linux alternative to MS Exchange so that all the features of outlook work?

12 14

[Dovecot] doveadm fts optimize undocumented
by Dave Abrahams 04 Dec '12

04 Dec '12

Subject says all. Timo, would you write something up where the world will find it? Also, it would be great if doveadm fts was covered in the doveadm manpage. Thanks, -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

2 6

[Dovecot] Maidir layout : why and how ?
by Thomas Hummel 04 Dec '12

04 Dec '12

Hello Timo, I've been using dovecot from many years now using Maildir mailbox format with default layout (Maildir++, everthing in a single directory). What would be the reason(s) to convert to Layout=fs ? I can only think of one : to allow dots in mailbox names. And even in that case, I guess I still have the option to use the litescape plugin. Besides, how would one process to do the conversion ? Using dsync ? Thanks. -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Groupe Exploitation et Infrastructure

2 1

[Dovecot] CVE-2012-5620
by Timo Sirainen 04 Dec '12

04 Dec '12

This CVE isn't worthy of existence. A user can crash his/her own IMAP session by issuing a specific SEARCH command, which is just slightly worse than user issuing a LOGOUT command. It took years for people to notice this bug, because just about no IMAP client issues such a command.

1 0

[Dovecot] imap-login hanging when firewall blocks ssl handshaking
by Erik A Johnson 04 Dec '12

04 Dec '12

imap-login processes are hanging (using 100% of CPU) when connected from a client that is partially blocked by a firewall. It appears that imap-login is stuck in a loop trying to complete an ssl handshake. imap-login is working fine for other clients not blocked by the firewall (including localhost). This is dovecot 2.1.10 under Mac OS X 10.8.2 (compiled from sources); the firewall is Little Snitch 3.0.1 blocking port 993, which appears to let the connection initiate but then squashes and disconnects the socket during ssl handshaking. gdb backtrace and Activity Monitor's "Sample Process" show that imap-login is stuck calling ioloop-kqueue's io_loop_handler_run -> io_loop_call_io -> ssl_step repeatedly; dtruss shows that it is repeatedly making system calls to kevent and read, the latter returning -1 with errno 57=ENOTCONN="Socket is not connected". (I also tried ./configure --with-ioloop=poll and --with-iopoll=select instead of the default best = kqueue but the results were the same; --with-iopoll=epoll didn't work because epoll is not available on this machine.) The client, initiated by the command "openssl s_client -connect SERVER:993", first responds "CONNECTED(00000003)" but then immediately the error "60278:error:140790E5:SSL routines:SSL23_WRITE:ssl handshake failure:/SourceCache/OpenSSL098/OpenSSL098-44/src/ssl/s23_lib.c:182:". The infinite loop is in src/lib/ioloop.c in the function "io_loop_run" where the statement "while (ioloop->running) io_loop_handler_run(ioloop)" is executed. While I think the firewall is not doing the right thing (i.e., it lets a connection start but then kills it), it seems that imap-login should fail gracefully when the socket is no longer connected. Ideas? Thanks. ----------------------------------- /usr/local/sbin/dovecot -n # 2.1.10: /usr/local/etc/dovecot/dovecot.conf # OS: Darwin 12.2.0 x86_64 default_internal_user = _dovecot default_login_user = _dovenull namespace { hidden = no list = yes location = mbox:/Users/%u/Documents/Mailboxes prefix = mbox/ separator = / type = private } namespace { inbox = yes location = maildir:/Users/%u/Documents/Maildir mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { driver = pam } protocols = imap lmtp ssl = required ssl_cert = </usr/local/etc/dovecot/dovecot.pem ssl_key = </usr/local/etc/dovecot/dovecot.pem userdb { driver = passwd } protocol imap { mail_plugins = " zlib imap_zlib" } ----------------------------------- gdb's "bt full" gives various results depending on where in the loop, but they are they same from ssl23_accept on down. (gdb) bt full #0 0x00007fff92d19f63 in sha1_block_data_order () #1 0x00007fff92d1a4ae in SHA1_Final () #2 0x00007fff92d45430 in EVP_DigestFinal_ex () #3 0x00007fff92d17d7a in ssleay_rand_add () #4 0x00007fff974529be in ssl23_accept () #5 0x000000010cac0729 in ssl_handshake [inlined] () at /private/tmp/johnsone/dovecot-2.1.10/src/login-common/ssl-proxy-openssl.c:465 ret = -949939536 #6 0x000000010cac0729 in ssl_step (proxy=0x7fff53152b90) at ssl-proxy-openssl.c:528 ret = -949939536 #7 0x000000010cb11bd1 in io_loop_call_io (io=0x7f7fa0c1c240) at ioloop.c:379 ioloop = (struct ioloop *) 0x7f7fa0c03b00 t_id = 2 #8 0x000000010cb12c69 in io_loop_handler_run (ioloop=0x0) at ioloop-kqueue.c:148 tv = { tv_sec = 130, tv_usec = 124797 } ctx = (struct ioloop_handler_context *) 0x7f7fa0c1a890 ret = 1 ts = { tv_sec = 130, tv_nsec = 124797000 } events = (struct kevent *) 0x10cb4ee78 #9 0x000000010cb11b88 in io_loop_run (ioloop=0x7fff53152b90) at ioloop.c:398 No locals. #10 0x000000010cafde9b in master_service_run (service=0x7f7fa0c039f0, callback=0x7f7fa0c1c2d8) at master-service.c:543 No locals. #11 0x000000010cabdcd1 in main_deinit [inlined] () at /private/tmp/johnsone/dovecot-2.1.10/src/login-common/main.c:406 set_pool = (pool_t) 0x7f7fa100b420 login_socket = 0x10cb4f7f8 "?9??" #12 0x000000010cabdcd1 in login_binary_run (argc=1, argv=0x7f7fa0c037c0, binary=0x7fff53152b90) at main.c:407 set_pool = (pool_t) 0x7f7fa100b420 login_socket = 0x10cb4f7f8 "?9??" #13 0x00007fff8ac847e1 in start () (gdb) bt full #0 0x00007fff96e4c110 in malloc_zone_malloc () #1 0x00007fff96e4cba7 in malloc () #2 0x00007fff92d2c9e2 in CRYPTO_malloc () #3 0x00007fff92d4535f in EVP_DigestInit_ex () #4 0x00007fff92d17caa in ssleay_rand_add () #5 0x00007fff974529be in ssl23_accept () ... #0 0x00007fff94775ffa in read () #1 0x00007fff92cf41e6 in sock_read () #2 0x00007fff92d15598 in BIO_read () #3 0x00007fff9745294a in ssl23_read_bytes () #4 0x00007fff97452c3e in ssl23_get_client_hello () #5 0x00007fff97452b4d in ssl23_accept () ... -----------------------------------

4 14

[Dovecot] clucene (in)stability
by Dave Abrahams 04 Dec '12

04 Dec '12

Considering two crashing bugs, which have got no response (https://sourceforge.net/tracker/?func=detail&aid=3587525&group_id=80013&ati… and https://sourceforge.net/tracker/?func=detail&aid=3592291&group_id=80013&ati…) and the lack of activity in its Git repository, I am inclined to think that CLucene's current release series (e.g. 2.3.3.4) should be considered unsuitable for serious use (including with Dovecot). I'm going to try CLucene 0.9.21, which they claim has been "proven to be stable over time" (see http://clucene.sourceforge.net) and see how it works out. I wonder if there's a future envisioned for FTS with Dovecot that's suitable for small installations (i.e. doesn't require a Solr server) and has a good chance of correctness and stability? -- Dave Abrahams BoostPro Computing Software Development Training http://www.boostpro.com Clang/LLVM/EDG Compilers C++ Boost

2 2