After fixing the "duplicate compression" problem, we're now encountering
"imap-login: Error: proxy(USERNAME): connect(10.x.x.178, 993) failed: Cannot assign requested address (after 0 secs, local=10.x.x.100)"
in the logs. We already tried raising the ulimit, the max number of open files. Once we reach about 25k connections, we're getting the error above... for all local addresses. It seems as if the system cannot create any more outgoing connections.
We already optimized:
---- snip ---- net.ipv4.tcp_fin_timeout=5 # down from 30s
net.ipv4.tcp_tw_reuse=1 net.ipv4.tcp_tw_recycle=1 # http://redis4you.com/articles.php?id=012&name=Redis+high+traffic+connection+issue
net.ipv4.ip_local_port_range=10000 65000 # http://www.fromdual.com/huge-amount-of-time-wait-connections
net.ipv4.netfilter.ip_conntrack_max=524288 ---- snip ----
But still we get (once the load is rising beyond some point):
# fgrep "Cannot assign requested address" /var/log/dovecot/dovecot.log |awk '{print $NF}' | sort | uniq -c | sort -n 142 local=10.x.x.100) 147 local=10.x.x.107) 148 local=10.x.x.106) 151 local=10.x.x.104) 151 local=10.x.x.109) 152 local=10.x.x.105) 156 local=10.x.x.110) 162 local=10.x.x.102) 165 local=10.x.x.101) 178 local=10.x.x.103) 189 local=10.x.x.108)
We're using multiple local addresses when proxying to the backends
-- [*] sys4 AG
http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München
Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein