[Dovecot] outlook 2007 very slow.
Hello
I have a user with 2500+ sub folders. Total mailboxes size is around 6G. (mdbox, dovecot 2:2.0.14)
Syncing/Receiving appears to be slow, with outlook 2007. He does not want to switch to an alternative, due to various reasons.
I did not find any error logs indicate issues.
during idle, imap process appears to loop at adding inotify watches to all folders.
I found outlook-idle in wiki, but it is obsolete.
doveconf: Warning: Obsolete setting in /etc/dovecot/conf.d/20-imap.conf:55: imap_client_workarounds=outlook-idle is no longer necessary
Any one else having similar issue? Anything else I should do to narrow down the issue?
Thanks KuiZ
` Kui Zhang wrote:
Hello
I have a user with 2500+ sub folders. Total mailboxes size is around 6G. (mdbox, dovecot 2:2.0.14)
Syncing/Receiving appears to be slow, with outlook 2007. He does not want to switch to an alternative, due to various reasons.
Any one else having similar issue? Anything else I should do to narrow down the issue?
I can't speak for outlook 2007, but back in outlook 2000, as well asoutlook 2002, it spoke a broken dialect of IMAP that would cause it to hang if you enabled it to read multiple mailboxes at one time.
The only safe way I found to use it was to only let it use 1 connection at a time, and even then it wasn't impossible to cause to to fail.
Perhaps MS limited outlook to only 1 connection to IMAP servers -- when I spoke to the engineer, they said that really had IMAP support at the lowest level, as it allowed the use of non-MS servers and mail servers -- and they only wanted to support Exchange (in order to get sites to buy exchange!)...
The issue was reported broken in 2000, and they had not fixed it by2002 (office XP), so I moved to thunderbird...
I missed a few-several features, but I didn't miss the slowness andunreliability in everyday reading of email.
Another problem -- AFAIK, outlook is only 32bit. My mom getsharassed, constantly to move things out of her primary .pst file and into 'archives', (where she can't easily access them and they don't have to be indexed...) because, the internal format became more strained as it got larger. With 6G of folders, indexing those, your user might be hitting outlook memory problems (not running out, but 'thrashing')...
If possible, he might try unsubbing to older boxes on his mainaccount, and setup an alternate account to 'go into the archives'...that way syncing only with currently active folders should go much faster)...
Send him my condolences...
-l
Thanks KuiZ
On Mon, Sep 19, 2011 at 4:03 PM, Linda Walsh dovecot@tlinx.org wrote:
` Kui Zhang wrote:
Hello
I have a user with 2500+ sub folders. Total mailboxes size is around 6G. (mdbox, dovecot 2:2.0.14)
Syncing/Receiving appears to be slow, with outlook 2007. He does not want to switch to an alternative, due to various reasons.
Any one else having similar issue? Anything else I should do to narrow down the issue?
I can't speak for outlook 2007, but back in outlook 2000, as well as outlook 2002, it spoke a broken dialect of IMAP that would cause it to hang if you enabled it to read multiple mailboxes at one time.
The only safe way I found to use it was to only let it use 1 connection at a time, and even then it wasn't impossible to cause to to fail.
Perhaps MS limited outlook to only 1 connection to IMAP servers -- when I spoke to the engineer, they said that really had IMAP support at the lowest level, as it allowed the use of non-MS servers and mail servers -- and they only wanted to support Exchange (in order to get sites to buy exchange!)...
I thought it might have been something anti-competitive...
We decided to give outlook 2k10 a try. Everything appears to work so far. It seems to be using only 1 connection... 2k7 was using 5 connections, with multiple connections in idle state(adding inotify watches)
The issue was reported broken in 2000, and they had not fixed it by 2002 (office XP), so I moved to thunderbird...
thunderbird does not really work for us, due to amount of emails per mailbox. It was hogging all the memory + cpu.
Trying out claw-mail. It is working really well.
I missed a few-several features, but I didn't miss the slowness and unreliability in everyday reading of email.
Another problem -- AFAIK, outlook is only 32bit. My mom gets harassed, constantly to move things out of her primary .pst file and into 'archives', (where she can't easily access them and they don't have to be indexed...) because, the internal format became more strained as it got larger. With 6G of folders, indexing those, your user might be hitting outlook memory problems (not running out, but 'thrashing')...
If possible, he might try unsubbing to older boxes on his main account, and setup an alternate account to 'go into the archives'...that way syncing only with currently active folders should go much faster)...
Send him my condolences...
-l
Thanks KuiZ
On 20/09/2011 03:10, Kui Zhang wrote:
thunderbird does not really work for us, due to amount of emails per mailbox. It was hogging all the memory + cpu.
I think if you disable the new local indexing features in TB then it should start running fairly decently?
I don't have mega large inboxes, but basically no real concerns with my normal inboxes which are around the 44K level at the largest and the remainder around 10K-15K
Obviously it's a whole new problem to push out company wide configuration with the local indexing/download stuff disabled, but I think there is some ability to do this in newer versions?
Good luck
Ed W
On 2011-09-27 2:06 PM, Ed W lists@wildgooses.com wrote:
On 20/09/2011 03:10, Kui Zhang wrote:
thunderbird does not really work for us, due to amount of emails per mailbox. It was hogging all the memory + cpu.
I think if you disable the new local indexing features in TB then it should start running fairly decently?
Also - Outlook is by far much slower than Thunderbird in my experience...
--
Best regards,
Charles
I think if you disable the new local indexing features in TB then it should start running fairly decently?
I had indexing disabled... that did not help much.
TB work better after I have these settings...
mail.imap.expunge_after_delete true mail.imap.expunge_option 2 mail.server.default.autosync_offline_stores false mail.server.default.offline_download false mail.server.default.autosync_max_age_days 14
I think mail.imap.expunge_after_delete might have caused mdbox limit problem I had before... but not confirmed.
Also - Outlook is by far much slower than Thunderbird in my experience...
Multiple people in the office report outlook is faster (when it works).
KuiZ
On Tue, Sep 27, 2011 at 11:21 AM, Charles Marcus CMarcus@media-brokers.com wrote:
On 2011-09-27 2:06 PM, Ed W lists@wildgooses.com wrote:
On 20/09/2011 03:10, Kui Zhang wrote:
thunderbird does not really work for us, due to amount of emails per mailbox. It was hogging all the memory + cpu.
I think if you disable the new local indexing features in TB then it should start running fairly decently?
Also - Outlook is by far much slower than Thunderbird in my experience...
--
Best regards,
Charles
On Tue, 27 Sep 2011 14:50:39 -0700 Kui Zhang articulated:
On Tue, Sep 27, 2011 at 11:21 AM, Charles Marcus CMarcus@media-brokers.com wrote:
On 2011-09-27 2:06 PM, Ed W lists@wildgooses.com wrote:
On 20/09/2011 03:10, Kui Zhang wrote:
thunderbird does not really work for us, due to amount of emails per mailbox. It was hogging all the memory + cpu.
I think if you disable the new local indexing features in TB then it should start running fairly decently?
Also - Outlook is by far much slower than Thunderbird in my experience...
I think if you disable the new local indexing features in TB then it should start running fairly decently?
I had indexing disabled... that did not help much.
TB work better after I have these settings...
mail.imap.expunge_after_delete true mail.imap.expunge_option 2 mail.server.default.autosync_offline_stores false mail.server.default.offline_download false mail.server.default.autosync_max_age_days 14
I think mail.imap.expunge_after_delete might have caused mdbox limit problem I had before... but not confirmed.
Also - Outlook is by far much slower than Thunderbird in my experience...
Multiple people in the office report outlook is faster (when it works).
I have always found Outlook to be much faster than TB. In any case, Outlook 2007 is an old version. I am using the 2010 version at work and it is a much more polished application than the 2010 version and far superior to TB.
-- Jerry ✌ Dovecot.user@seibercom.net
Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.
On 2011-09-27 6:42 PM, Jerry dovecot.user@seibercom.net wrote:
I have always found Outlook to be much faster than TB. In any case, Outlook 2007 is an old version. I am using the 2010 version at work and it is a much more polished application than the 2010 version and far superior to TB.
That's funny - I find Outlooks email UI to be horrible. And HTML support relies on Word's HTML rendering engine? Give me a break...
Outlook's calendar is definitely superior, but as an email client it is only useful when it is used in a full blown Exchange environment. As an IMAP client, it blows chunks.
--
Best regards,
Charles
On Wed, 28 Sep 2011 09:03:18 -0400 Charles Marcus articulated:
On 2011-09-27 6:42 PM, Jerry dovecot.user@seibercom.net wrote:
I have always found Outlook to be much faster than TB. In any case, Outlook 2007 is an old version. I am using the 2010 version at work and it is a much more polished application than the 2010 version and far superior to TB.
That's funny - I find Outlooks email UI to be horrible. And HTML support relies on Word's HTML rendering engine? Give me a break...
Outlook's calendar is definitely superior, but as an email client it is only useful when it is used in a full blown Exchange environment. As an IMAP client, it blows chunks.
Like all things, the usefulness of any application can only be truly measured in the context of the end user's environment. I virtually never use HTML e-mail myself. There are a few publications that I subscribe to that supply their material in HTML format; however, they all also list a URL to view the material. I prefer to use that method instead. E-mail, in my opinion, is a poor environment for HTML.
MS Outlook's calender is the best available. I find Outlook's interface easy to use. Then again, I am quite familiar with it so that would only be natural.
You fail to mention what version of Outlook you are referring to so there is no way I can gather any useful data from your analysis other to state that I have never used any version of TB that I found as useful as a comparative version of Outlook. If it is Outlook 2007, then perhaps this comparison of products should be restricted to a four year old version of TB also. I think I can safely say without fear of contradiction that, that is not something anyone would readily want to do.
In any case, the idiom, "better the devil you know than the devil you don't" is apropos to this thread.
-- Jerry ✌ Dovecot.user@seibercom.net
Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header.
On 28/09/2011 15:04, Jerry wrote:
MS Outlook's calender is the best available. I find Outlook's interface easy to use. Then again, I am quite familiar with it so that would only be natural.
I have been waiting for what is likely to be TB 8 + the subsequent release of SoGo. I have some hope that the sogo plugins, which should then have all the patches they need natively in TB, will go a long way to level the playing field on the calendering vs Outlook... Watch this space and all that...
Just chewing the cud and all that, but one of the features which does very much annoy me about Outlook is the apparent failure to do simple nested indents on replied emails, ie like this one where you get each person's reply indented one level all the way down the exchange. I get so many emails where the user starts writing in various coloured writing to try and and make up for the completely flat reply... I *believe* this is entirely down to the settings you pick in Outlook, but it's clearly a common setup to have replies non indented?
(Plenty of things tick me off about TB, but this margin too small to accomodate them....)
Cheers
Ed W
On 9/27/2011 4:50 PM, Kui Zhang wrote:
I think if you disable the new local indexing features in TB then it should start running fairly decently?
I had indexing disabled... that did not help much.
TB work better after I have these settings...
mail.imap.expunge_after_delete true mail.imap.expunge_option 2 mail.server.default.autosync_offline_stores false mail.server.default.offline_download false mail.server.default.autosync_max_age_days 14
I think mail.imap.expunge_after_delete might have caused mdbox limit problem I had before... but not confirmed.
What, exactly, was the nature of the performance problem you originally mentioned to start this thread, the mailbox with the thousands of sub folders?
With GLODA and local synchronization disabled, using 1 IMAP connection instead of the default 5, disabling IDLE and using check interval seconds, and using the default:
mail.server.default.check_all_folders_for_new FALSE
then you should have excellent performance with TB regardless of the number of folders in a mailbox. Unless maybe the hardware or net pipe are lacking.
What are the specs of the client machine in question? What CPU/freq. Maybe more importantly, what is the link speed of the network between this PC and the Dovecot server? LAN or WAN?
-- Stan
On Tue, Sep 27, 2011 at 10:42 PM, Stan Hoeppner stan@hardwarefreak.com wrote:
On 9/27/2011 4:50 PM, Kui Zhang wrote:
I think if you disable the new local indexing features in TB then it should start running fairly decently?
I had indexing disabled... that did not help much.
TB work better after I have these settings...
mail.imap.expunge_after_delete true mail.imap.expunge_option 2 mail.server.default.autosync_offline_stores false mail.server.default.offline_download false mail.server.default.autosync_max_age_days 14
I think mail.imap.expunge_after_delete might have caused mdbox limit problem I had before... but not confirmed.
What, exactly, was the nature of the performance problem you originally mentioned to start this thread, the mailbox with the thousands of sub folders?
This thread might be getting little off topic.
It was for inotify loop on server side during idle (outlook 2k7), with approximate 2.1k folders.
With GLODA and local synchronization disabled, using 1 IMAP connection instead of the default 5, disabling IDLE and using check interval seconds, and using the default:
I have not notice looping on idle when client using Thunderbird. So it might be outlook specific.
mail.server.default.check_all_folders_for_new FALSE
Not feasible. Few folders need to be checked periodically...
then you should have excellent performance with TB regardless of the number of folders in a mailbox. Unless maybe the hardware or net pipe are lacking.
Ya, but TB would be doing less work.
What are the specs of the client machine in question? What CPU/freq. Maybe more importantly, what is the link speed of the network between this PC and the Dovecot server? LAN or WAN?
avg ttl = 0.5ms Sustain 10-12MB/s, concurrent connections, from 5+ workstations.
No load issues recorded on the server side.
The clients have 8GB of ram, athlon II X4 640 quad core. they should have enough juice...
On an athlon 3200, 2G Ram, I did some tests on an account, with 3GB on disk, approximate 100k emails, and 30 folders.
on the client side, when no mail client running. 450-500MB ram used. CPU at 1-2%
TB hangs on start up, for extent period of time. cpu at 100%, ram at 1 thunderbird people.
- 1.2 GB used. CPU usage almost always at 100%. And it hangs from time to time. The client side disk usage for TB is around 200MB… why would it need 500MB of ram? This is something I will bring up with the
With claws-mail, cpu goes up depending on amount of email in the folder. And cpu usage drop almost immediately after folder switch. around 480 – 520 MB ram used. So the client box is not too slow.
KuiZ
-- Stan
On 28/09/2011 21:00, Kui Zhang wrote:
TB hangs on start up, for extent period of time. cpu at 100%, ram at 1 thunderbird people.
- 1.2 GB used. CPU usage almost always at 100%. And it hangs from time to time. The client side disk usage for TB is around 200MB… why would it need 500MB of ram? This is something I will bring up with the
If you care to debug in more detail, you may learn a lot by watching the network traffic at this point? You can setup debugging on the server side, but personally I find this a touch hard to setup for one off sessions (and shared IPs/mailboxes, etc). Also consider wireshark and just tracing a single machine. The point being to see if it's locked up because it's thrashing the mail server for some reason, or if it's doing something silly client side?
Random untested ideas: the ping speed to the server (something like 100-200 round trips per sec
- I believe it pulls the folder list down at startup. With thousands of folders in your case (did I understand that?) you might find it's doing some silly select on each folder and hence spending ages being bound by
max I think you said?), or perhaps it's even worse than that if it causes some disk seek for each folder?
Quantity of headers could be large under certain circumstances - check if you are network bandwidth bound?
TB might be doing something silly locally and you are bound by disk seek time on your local machine as it does whatever it does to several thousand mbox files? Move the TB local folder to some slower/faster disk and observe if the startup speed gets proportionally slower/faster..? Eg I slapped in some large flash drive to my Mac and now I keep forgetting that others still have seek time limitations starting apps...
Good luck - interested to hear if you can trace this to something?
Ed W
P.S. I will try and post some tips in a new thread, but I found that TB and other clever clients can benefit enormously if you turn on the appropriate zlib stuff that means the COMPRESS extension is supported (not on by default). Outlook hasn't historically supported this, so I doubt it will help above, but it's one feature that can give TB the edge over Outlook.
On 2011-09-19 7:03 PM, Linda Walsh dovecot@tlinx.org wrote:
Another problem -- AFAIK, outlook is only 32bit. My mom gets harassed, constantly to move things out of her primary .pst file and into 'archives', (where she can't easily access them and they don't have to be indexed...) because, the internal format became more strained as it got larger. With 6G of folders, indexing those, your user might be hitting outlook memory problems (not running out, but 'thrashing')...
Outlook < 2007 were limited to 2GB .pst files.
2007 could use max of 20GB .pst files, although in my experience it gets slow/sluggish with more than 10GB (and keep good backups, .pst files can get corrupted if you look at them too hard)...
IMAP support got much better in 2007, and is even better in 2010, but Outlook still only really shines as an Exchange client, and that is the way Microsoft wants it (and understandably so)...
Personally, I would only use Outlook in an Exchange environment, never as a standalone email client, although it's calendar is much better than Thunderbird+Lightning I'm sad to say...
--
Best regards,
Charles
On 20/09/11 11:21, Charles Marcus wrote:
IMAP support got much better in 2007,
How did it get better? They removed NAMESPACE support when moving from 2003 to 2007, which was a complete ball ache for us.
and is even better in 2010
Do they support ACL in 2010? They don't in 2007, which is really frustrating. What is better about 2010 IMAP support compared to 2007? I'm genuinely interested to know...
-- Mike Cardwell https://grepular.com/ https://twitter.com/mickeyc Professional http://cardwellit.com/ http://linkedin.com/in/mikecardwell PGP.mit.edu 0018461F/35BC AF1D 3AA2 1F84 3DC3 B0CF 70A5 F512 0018 461F
On 2011-09-20 6:30 AM, dovecot@lists.grepular.com dovecot@lists.grepular.com wrote:
On 20/09/11 11:21, Charles Marcus wrote:
IMAP support got much better in 2007,
How did it get better? They removed NAMESPACE support when moving from 2003 to 2007, which was a complete ball ache for us.
2003 was basically unusable on large mail stores. 2007 actually became usable, in that it was much more responsive when working with mail, but my testing didn't really go beyond that, it was more curiosity than anything (and, I have ONE friend who insists on using Outlook because the Calendar is so much better than anything else out there).
and is even better in 2010
Do they support ACL in 2010? They don't in 2007, which is really frustrating. What is better about 2010 IMAP support compared to 2007? I'm genuinely interested to know...
What do you mean by 'ACL support'? Do you mean the ability to create/edit them on shared folders? If so, then no, not natively, I don't think, but I didn't do any intensive testing, all I can tell you is it is much more responsive, especially on larger folders.
--
Best regards,
Charles
On 20/09/11 11:46, Charles Marcus wrote:
and is even better in 2010
Do they support ACL in 2010? They don't in 2007, which is really frustrating. What is better about 2010 IMAP support compared to 2007? I'm genuinely interested to know...
What do you mean by 'ACL support'? Do you mean the ability to create/edit them on shared folders?
Yes. Support for the IMAP ACL extension.
If so, then no, not natively, I don't think, but I didn't do any intensive testing, all I can tell you is it is much more responsive, especially on larger folders.
This is a real shame. IMAP can do so much more than Exchange's IMAP implementation allows.
-- Mike Cardwell https://grepular.com/ https://twitter.com/mickeyc Professional http://cardwellit.com/ http://linkedin.com/in/mikecardwell PGP.mit.edu 0018461F/35BC AF1D 3AA2 1F84 3DC3 B0CF 70A5 F512 0018 461F
participants (8)
-
Charles Marcus
-
dovecot＠lists.grepular.com
-
Ed W
-
Jerry
-
Kui Zhang
-
Linda Walsh
-
Mike Cardwell
-
Stan Hoeppner