Slow performance with large folders over the Internet
Dovecot package version is 1:1.2.15-7+deb6u1. It is in Debian 6.0.10, using the Debian package.
The server is in my basement at home, and is exposed to the Internet so I can fully access my mail from anywhere. I use IMAP for reading mail.
I have a number of folders in my mailbox that have thousands of messages in them, from mailing lists.
When I'm at home, I have a LAN connection to the server. It goes through a Cisco firewall that limits the connection speed to 100Mb/s. In this situation, I can open a folder with 25000 messages in it, click on the next unread message that Thunderbird did not know about before, and within a second or two, the message will download, allowing me to view it and reply.
When I'm at work, with highly variable network latency between Thunderbird and the server, doing exactly the same thing takes a LOT longer. I have seen it take as long as 15 minutes for a single message. If I open a folder with only a few messages in it, it is fast.
The server is not overloaded -- I can log into it with ssh and use "mutt -f" to open a folder directly. Loading thousands of messages into mutt takes a while, but I have no difficulty using the ssh connection and running commandline programs.
This suggests that the IMAP communication between the server and the client involves a large amount of back and forth communication when the message count in the folder is high, possibly something for every message in the folder. It happens quickly on a LAN but crawls on a connection with high latency. I can understand it taking a few seconds longer on a high-latency link, but it takes minutes.
I do plan on building a new server and migrating to Dovecot 2.x, but I haven't had the time to work on that.
Is this a known problem? If so, is it fixed in 2.x?
Thanks, Shawn
On 03/31/2017 12:03 AM, Shawn Heisey wrote:
Dovecot package version is 1:1.2.15-7+deb6u1. It is in Debian 6.0.10, using the Debian package.
The server is in my basement at home, and is exposed to the Internet so I can fully access my mail from anywhere. I use IMAP for reading mail.
I have a number of folders in my mailbox that have thousands of messages in them, from mailing lists.
When I'm at home, I have a LAN connection to the server. It goes through a Cisco firewall that limits the connection speed to 100Mb/s. In this situation, I can open a folder with 25000 messages in it, click on the next unread message that Thunderbird did not know about before, and within a second or two, the message will download, allowing me to view it and reply.
When I'm at work, with highly variable network latency between Thunderbird and the server, doing exactly the same thing takes a LOT longer. I have seen it take as long as 15 minutes for a single message. If I open a folder with only a few messages in it, it is fast.
The server is not overloaded -- I can log into it with ssh and use "mutt -f" to open a folder directly. Loading thousands of messages into mutt takes a while, but I have no difficulty using the ssh connection and running commandline programs.
This suggests that the IMAP communication between the server and the client involves a large amount of back and forth communication when the message count in the folder is high, possibly something for every message in the folder. It happens quickly on a LAN but crawls on a connection with high latency. I can understand it taking a few seconds longer on a high-latency link, but it takes minutes.
I do plan on building a new server and migrating to Dovecot 2.x, but I haven't had the time to work on that.
Is this a known problem? If so, is it fixed in 2.x?
Thanks, Shawn
Hi Shawn,
If you think that imap is the problem, you can do an imap session by hand and see where the problems are:
openssl s_client -CApath /path/to/your/certs -connect your.server:143 -starttls imap
See fi. http://wiki.linuxquestions.org/wiki/Testing_IMAP_via_telnet
But from your mail I would say that you might have networking or firewall issues. So I would be looking for interface errors, missing ping packets, traceroute output and so on. Best,
gerard
On 03/31/2017 12:03 AM, Shawn Heisey wrote:
Dovecot package version is 1:1.2.15-7+deb6u1. It is in Debian 6.0.10, using the Debian package.
The server is in my basement at home, and is exposed to the Internet so I can fully access my mail from anywhere. I use IMAP for reading mail.
I have a number of folders in my mailbox that have thousands of messages in them, from mailing lists.
When I'm at home, I have a LAN connection to the server. It goes through a Cisco firewall that limits the connection speed to 100Mb/s. In this situation, I can open a folder with 25000 messages in it, click on the next unread message that Thunderbird did not know about before, and within a second or two, the message will download, allowing me to view it and reply.
When I'm at work, with highly variable network latency between Thunderbird and the server, doing exactly the same thing takes a LOT longer. I have seen it take as long as 15 minutes for a single message. If I open a folder with only a few messages in it, it is fast.
The server is not overloaded -- I can log into it with ssh and use "mutt -f" to open a folder directly. Loading thousands of messages into mutt takes a while, but I have no difficulty using the ssh connection and running commandline programs.
This suggests that the IMAP communication between the server and the client involves a large amount of back and forth communication when the message count in the folder is high, possibly something for every message in the folder. It happens quickly on a LAN but crawls on a connection with high latency. I can understand it taking a few seconds longer on a high-latency link, but it takes minutes.
I do plan on building a new server and migrating to Dovecot 2.x, but I haven't had the time to work on that.
Is this a known problem? If so, is it fixed in 2.x?
Thanks, Shawn This sounds like your companies firewall trying a mitm attack or similar. Just a wild guess.
If the SSH-connection is good (probably ignored by the firewall or maybe even prioritized), then forward your IMAP-traffic through it and see if the problem persists. This is not meant as a solution, but to help analyze the problem.
# ssh -L 10993:127.0.0.1:993 you@your.server Then connect with Thunderbird to 127.0.0.1:10993. You could also use :143, the SSH-tunnel is already encrypted.
Greetings Daniel
participants (3)
-
Daniel Tröder
-
Gerard Ranke
-
Shawn Heisey