On 8/19/2013 7:03 AM, Reindl Harald wrote:

Am 19.08.2013 14:00, schrieb LuKreme:

...

All of a sudden I am getting these errors on one of my accounts:

imap-login: Info: Maximum number of connections from user+IP exceeded (mail_max_userip_connections=10) It was working fine last night when I went to bed, and is posting these errors nearly constantly

in case of IMAP 10 is *way* too low!

keep in mind that

• a IMAP client opens one connection *per folder*

What do you mean by "per folder"? I've been limiting Tbird to 2 IMAP connections for many years and, unsurprisingly, it never opens more than two IMAP connections to Dovecot no matter how many folders I access, tabs I have open, or searches I perform, etc:

tcp 0 0 192.168.100.9:143 192.168.100.53:1663 ESTABLISHED 13189/imap tcp 0 0 192.168.100.9:143 192.168.100.53:1672 ESTABLISHED 13192/imap

And with the default TB limit of 5 it never opens more than 5. Which clients exhibit this "per folder" connection behavior? That seems totally unnecessary.

• if you have 5 folders and a user with 3 devices (workstation, phone, tablet) you are done

Again, not folder dependent but client configuration dependent. If your client is RC it never opens more than one connection per user, and closes the connection after each operation.

• if you have a few imap-users behind the same NAT you are done

This isn't correct either. It's user+IP. So you could have 30 connections from 3 users, 100 from 10 users, through one NAT IP, with a setting of 10.

-- Stan

On 8/19/2013 4:10 PM, Reindl Harald wrote:

may i suggest you read about how IMAP IDLE works?

Oh, well sure, if you hang your hat on IDLE then your arguments here might make sense. But because of the brain dead one socket per folder architecture of IDLE few have adopted it en masse. Which is why my comments ignored the existence of IDLE. And which is also why the creators of the RFC stated clients must not count on the existence of IDLE and must poll, which seems really odd. Many have, and still ask, why even have IDLE then if we must still poll?

http://tools.ietf.org/html/rfc2177

"(While the spec actually does allow a server to push EXISTS responses aysynchronously, a client can't expect this behaviour and must poll.)"

Given the option of potentially dozens of open sockets between his server and any client simply to allow IDLE to work for all folders, or one or two connections and strictly client polling, I'd guess most admins will choose the latter.

-- Stan

On 8/19/2013 6:55 PM, Reindl Harald wrote:

Am 20.08.2013 01:45, schrieb Stan Hoeppner:

...

On 8/19/2013 4:10 PM, Reindl Harald wrote:

...

may i suggest you read about how IMAP IDLE works?

Oh, well sure, if you hang your hat on IDLE then your arguments here might make sense. But because of the brain dead one socket per folder architecture of IDLE few have adopted it en masse. Which is why my comments ignored the existence of IDLE. And which is also why the creators of the RFC stated clients must not count on the existence of IDLE and must poll, which seems really odd. Many have, and still ask, why even have IDLE then if we must still poll?

http://tools.ietf.org/html/rfc2177

"(While the spec actually does allow a server to push EXISTS responses aysynchronously, a client can't expect this behaviour and must poll.)"

Given the option of potentially dozens of open sockets between his server and any client simply to allow IDLE to work for all folders, or one or two connections and strictly client polling, I'd guess most admins will choose the latter

why we have IDLE is easy explained, i get around 500 mails per day well, i can't imagine my personal work-load woking without IDLE

30 folders sorted with Sieve

• several lists with own folders
• company (there folders, one for internal lists)
• customers
• vendors
• server-status (logwatch, mail-stats of 20 servers)
• error-notifies from watchdog (own cron-watchdogs, HP ILO, VMware vSphere, UPS...)

INBOX is a place where rarely a message comes in and with K9 on Android it's easy to select which folders should be considered for the common-inbox and which are pointless on a mobile (INBOX is none of them)

IDLE is not required for this. Polling, which is the default on all MUAs, accomplishes the same over one socket, a few max, depending on what you're doing -concurrently- in the MUA.

on a mailserver which can handle thousands of connections there is rarely a reason to disable IDLE and so a connection limit of 10 per IP is questionable

The server resources aren't necessarily a problem as you can always go cluster. One potential problem though, and there are likely others, is that you're potentially increasing the SPI/NAT session tracking on the edge router by 3-6 fold by allowing 30 sessions vs 5 or 10. Add that on top of the other traffic types and, for many, this may require larger routers, a license upgrade, or both. If you're an org of any size and tunneling the IMAP sessions through VPN routers, an upgrade would likely be mandatory.

Thus for some orgs simply increasing allowed connections to support IDLE on arbitrary folder counts may come with a $20-100K price tag. If this was money in your pocket, would you spend it to simply replace poll with push, given that poll works fine, and given that push yields no -real- advantage over poll?

-- Stan

On 8/20/2013 1:47 AM, Ben wrote:

...

IDLE is not required for this. Polling, which is the default on all MUAs,

Not entirely sure I'd call it "the default", Thunderbird seems to setup for IDLE as default these days, and I suspect Apple Mail and any of the other popular ones do too.

The source code probably looks something like if (canIdle) then {do stuff} else {do polling}

I haven't looked at a recent fresh install config of TB. If they have started enabling IDLE by default it's recently, and they'd have been required to increase the default 5 cached connections. Otherwise many folk would never see new mail notification on some of their server side sorted folders. And Joe and Susie Sixpack have no knowledge of the relationship between connection (socket) count and IDLE requirements, and I'd guess no MUA has documentation explaining this. I'm pretty sure Mozilla does not. I'm sure there are forum posts about it, but I doubt official documentation exists.

For IDLE to work seamlessly requires basically 3 things:

1. MUA counts all IMAP folders and
2. Always maintains $folder_count sockets
3. IMAP servers must allow basically unlimited sockets

In a closed environment such as a corp/uni this can be achieved, and without unlimited sockets, if staff can enforce a limit on folder count. In an open environment such as an ISP or freemail provider, they'd have to allow unlimited sockets. Some may do so currently. I have no knowledge of what such providers are doing in this regard as I don't use them nor support anyone who does.

-- Stan

On 2013-08-19 23:00, Stan Hoeppner wrote:

...

• a IMAP client opens one connection *per folder*

What do you mean by "per folder"? I've been limiting Tbird to 2 IMAP connections for many years and, unsurprisingly, it never opens more than two IMAP connections to Dovecot no matter how many folders I access, tabs I have open, or searches I perform, etc:

tcp 0 0 192.168.100.9:143 192.168.100.53:1663 ESTABLISHED 13189/imap tcp 0 0 192.168.100.9:143 192.168.100.53:1672 ESTABLISHED 13192/imap

And with the default TB limit of 5 it never opens more than 5. Which clients exhibit this "per folder" connection behavior? That seems totally unnecessary.

Any client which supports the 'IDLE' command does this; it's a mechanism to avoid that a client has to poll the IMAP server for new mail. The client does an 'IDLE' call *per folder* which only returns when the server adds new mail to the folder. Hence, the IDLE call blocks the connection, which is why mail clients which use IDLE have to establish multiple IMAP connections, one per folder which is monitored using this feature.

-- Frerich Raabe - raabe@froglogic.com www.froglogic.com - Multi-Platform GUI Testing