[Dovecot] OT: Looking for a robust IMAP client
This weekend we had a runaway email endless loop. When it was killed after 18 hours, my inbox had 135,000 messages in it...there were two messages that were being endlessly sent and bounced and I'm on the postmaster alias. Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up. I ended up firing up Pine to do the final 65,000...whereas TBird had had mulitple imap sessions (and failed), Pine only had one and did the job. And even after I had done the mass delete of the other 65,000 and the inbox was down to 2000 messages, TBird was still hiding under the covers and telling me there were still 135,000 messages. In the end, I had to kill the TBird profile for that account and recreate it.
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
I'm wondering if there is something we can tell users to use when Things Are Dire. GUI would be better since it removes one of the few remaining reasons for a logon server
==== Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, Bard College, New York 12504 sdean@bard.edu voice: 845-758-7475, fax: 845-758-7035
Stewart Dean wrote:
This weekend we had a runaway email endless loop. When it was killed after 18 hours, my inbox had 135,000 messages in it...there were two messages that were being endlessly sent and bounced and I'm on the postmaster alias. Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up. I ended up firing up Pine to do the final 65,000...whereas TBird had had mulitple imap sessions (and failed), Pine only had one and did the job. And even after I had done the mass delete of the other 65,000 and the inbox was down to 2000 messages, TBird was still hiding under the covers and telling me there were still 135,000 messages. In the end, I had to kill the TBird profile for that account and recreate it.
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
I'm wondering if there is something we can tell users to use when Things Are Dire. GUI would be better since it removes one of the few remaining reasons for a logon server I think Evolution handles large bundles of mail really well - though on Windows it's a bit of a mess of an application. It doesn't do single large messages very quickly though.
Alternately, if you're using Maildir, you could always grep and rm.
Rick
- Stewart Dean sdean@bard.edu:
This weekend we had a runaway email endless loop. When it was killed
after 18 hours, my inbox had 135,000 messages in it...there were two
messages that were being endlessly sent and bounced and I'm on the
postmaster alias. Thunderbird was able to do a mass select of one of
the two messages, and deleted 65,000, but after that it locked up. I
ended up firing up Pine to do the final 65,000...whereas TBird had had
mulitple imap sessions (and failed), Pine only had one and did the job. And even after I had done the mass delete of the other 65,000 and the
inbox was down to 2000 messages, TBird was still hiding under the covers
and telling me there were still 135,000 messages. In the end, I had to
kill the TBird profile for that account and recreate it.Is there a simple robust IMAP client to replace Pine (which I *think* is
no longer supported)? GUI or TTY session?
I use mutt to do IMAP and SMTP and I like it. YMMV...
p@rick
-- state of mind Agentur für Kommunikation, Design und Softwareentwicklung
Patrick Koetter Tel: 089 45227227 Echinger Strasse 3 Fax: 089 45227226 85386 Eching Web: http://www.state-of-mind.de
Amtsgericht München Partnerschaftsregister PR 563
Stewart Dean schrieb:
This weekend we had a runaway email endless loop. When it was killed after 18 hours, my inbox had 135,000 messages in it...there were two messages that were being endlessly sent and bounced and I'm on the postmaster alias. Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up. I ended up firing up Pine to do the final 65,000...whereas TBird had had mulitple imap sessions (and failed), Pine only had one and did the job. And even after I had done the mass delete of the other 65,000 and the inbox was down to 2000 messages, TBird was still hiding under the covers and telling me there were still 135,000 messages. In the end, I had to kill the TBird profile for that account and recreate it.
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
alpine, the official successor of pine:
Hi
This weekend we had a runaway email endless loop. When it was killed after 18 hours, my inbox had 135,000 messages in it...there were two messages that were being endlessly sent and bounced and I'm on the postmaster alias.
By far the fastest option is to simply use a tool which can examine the file system and delete - eg SSH. (try winscp on windows if you want a gui...)
Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up. I ended up firing up Pine to do the final 65,000...whereas TBird had had mulitple imap sessions (and failed), Pine only had one and did the job. And even after I had done the mass delete of the other 65,000 and the inbox was down to 2000 messages, TBird was still hiding under the covers and telling me there were still 135,000 messages. In the end, I had to kill the TBird profile for that account and recreate it.
To be fair to tbird - probably what happened was that it was shuffling lots of delete commands to the backend server. When you "cheated" and used another tool, it was then doing it's sync to discover which messages were deleted. Unless you have a VERY fast internet connection then be aware that the commands it uses to check for new messages could actually take multiple minutes to execute and many more while it deletes from the local offline index.
Basically it's a good little program but not really optimised for this requirement
I'm wondering if there is something we can tell users to use when Things Are Dire. GUI would be better since it removes one of the few remaining reasons for a logon server
You might want to consider scripting a little perl micro interface? I saw someone once write a very basic curses mail client in perl (ruby, etc would also be fine, nothing magic about perl) and for the kind of use case you have in mind something like a "delete by user" or "delete by message_id" might be helpful?
Additionally consider adding some protection against mail loops to your server - it used to be de-riguer back in the procmail days, but something as simple as checking for an "X-Been-Here" on any auto generated content is a good first start. Your choice of MDA is likely to determine the exact favoured approach.
Auto "out of the office" responders should definitely only notify every X days as well to help with this kind of thing.
Finally checking for the mandatory headers that all auto generated stuff should create should help...
Good luck
Ed W
On 12/15/2008, Ed W (lists@wildgooses.com) wrote:
Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up.
I'd never try to delete that many at once...
It very likely wasn't locked up though, it probably was working furiously to try to do what you told it to do - the problem is, it can *appear* to be locked up, even for many minutes, but if you let it go, it will eventually finish (or time out)...
But, if you ever try this again, it helps a LOT if you do a 'SHIFT-delete' (press/hold the shift key, then tap the Delete button on the keyboard) - this bypasses the Trash - otherwise, it isn't deleting them it is MOVING them to the Trash, which can take a long time for that many messages.
I usually work with a thousand or so at a time if I need to do something like this, and it works, although it certainly isn't instantaneous...
--
Best regards,
Charles
Charles Marcus wrote:
On 12/15/2008, Ed W (lists@wildgooses.com) wrote:
Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up.
I'd never try to delete that many at once...
It very likely wasn't locked up though, it probably was working furiously to try to do what you told it to do - the problem is, it can *appear* to be locked up, even for many minutes, but if you let it go, it will eventually finish (or time out)...
Pine did it in2-3 minutes with one imapd instance; TBird was thrashing mightly for 20+ minutes with 4-8 imapd instances, and no progress in site....even after Pine had deleted the inbox down to 2000 messages.
But, if you ever try this again, it helps a LOT if you do a 'SHIFT-delete' (press/hold the shift key, then tap the Delete button on the keyboard) - this bypasses the Trash - otherwise, it isn't deleting them it is MOVING them to the Trash, which can take a long time for that many messages.
Was not moving to Trash, just directly expunging stuff that had been marked for deletion
I usually work with a thousand or so at a time if I need to do something like this, and it works, although it certainly isn't instantaneous...
With
-- ==== Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, Bard College, New York 12504 sdean@bard.edu voice: 845-758-7475, fax: 845-758-7035
On 12/15/2008 2:34 PM, Stewart Dean wrote:
Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up.
I'd never try to delete that many at once...
It very likely wasn't locked up though, it probably was working furiously to try to do what you told it to do - the problem is, it can *appear* to be locked up, even for many minutes, but if you let it go, it will eventually finish (or time out)...
Pine did it in2-3 minutes with one imapd instance; TBird was thrashing mightly for 20+ minutes with 4-8 imapd instances, and no progress in site....even after Pine had deleted the inbox down to 2000 messages.
I never said TBird was 'better' than Pine, I merely commented on how TBird works with lots of messages in my experience...
Also, from what you just said, you were working with that many messages with Pine at the same time as with TBird?
TBird is an excellent IMAP client, as long as you understand its quirks and work with them.
Bottom line - if you expect it to behave exactly like Pine - or like you think it *should* - then expect to be disappointed.
But, if you ever try this again, it helps a LOT if you do a 'SHIFT-delete' (press/hold the shift key, then tap the Delete button on the keyboard) - this bypasses the Trash - otherwise, it isn't deleting them it is MOVING them to the Trash, which can take a long time for that many messages.
Was not moving to Trash, just directly expunging stuff that had been marked for deletion
I don't know what that means in TBird-speak. There is no 'expunge' command that I am aware of, either as a toolbar button or a menu choice.
There is a 'delete' button on the toolbar, and you can select messages and hit the 'delete' key on the keyboard.
You can also right-click on the Trash and 'empty' it.
The only place I know of that contains the word 'expunge' is in the Account Settings, where you can tell it to expunge the Inbox on exit.
So, I'm curious - what, exactly, did you do in TBird? You selected all of the messages in the Inbox, then... ?
I usually work with a thousand or so at a time if I need to do something like this, and it works, although it certainly isn't instantaneous...
With
a mbox format inbox, I don't know that it matters much whether it's 10 files or 10,000...it's still gotta haul out the whole ugly thing.
Ok, well, I only use maildir format, so can't speak to TBirds performance or quirks wrt mbox...
--
Best regards,
Charles
Charles Marcus wrote:
On 12/15/2008 2:34 PM, Stewart Dean wrote:
Thunderbird was able to do a mass select of one of the two messages, and deleted 65,000, but after that it locked up.
I'd never try to delete that many at once...
It very likely wasn't locked up though, it probably was working furiously to try to do what you told it to do - the problem is, it can *appear* to be locked up, even for many minutes, but if you let it go, it will eventually finish (or time out)...
Pine did it in2-3 minutes with one imapd instance; TBird was thrashing mightly for 20+ minutes with 4-8 imapd instances, and no progress in site....even after Pine had deleted the inbox down to 2000 messages.
I never said TBird was 'better' than Pine, I merely commented on how TBird works with lots of messages in my experience...
Oh, I only use Pine as a last resort. Tbrid had done the first 65,000 just fine...and then it locked up
Also, from what you just said, you were working with that many messages with Pine at the same time as with TBird?
TBird is an excellent IMAP client, as long as you understand its quirks and work with them.
Been using it for years
Bottom line - if you expect it to behave exactly like Pine - or like you think it *should* - then expect to be disappointed.
I would like Tbird to do just what it does now, but be more robust (and maybe a little quicker) about it. Bombproof, as they say.
But, if you ever try this again, it helps a LOT if you do a 'SHIFT-delete' (press/hold the shift key, then tap the Delete button on the keyboard) - this bypasses the Trash - otherwise, it isn't deleting them it is MOVING them to the Trash, which can take a long time for that many messages.
Was not moving to Trash, just directly expunging stuff that had been marked for deletion
I don't know what that means in TBird-speak. There is no 'expunge' command that I am aware of, either as a toolbar button or a menu choice.
There is a 'delete' button on the toolbar, and you can select messages and hit the 'delete' key on the keyboard.
You can also right-click on the Trash and 'empty' it.
The only place I know of that contains the word 'expunge' is in the Account Settings, where you can tell it to expunge the Inbox on exit.
So, I'm curious - what, exactly, did you do in TBird? You selected all of the messages in the Inbox, then... ?
I marked them for Deletion (dunno what exactly that does, maybe something in that first mbox entry or in the index or....but it does NOT xfer them to another folder, just gives them a "black spot". Then, under file, I would select "Compact Folders", though now I use a TBird AddOn called Purge, which give me a control bar Icon to do it.
I usually work with a thousand or so at a time if I need to do something like this, and it works, although it certainly isn't instantaneous...
With
a mbox format inbox, I don't know that it matters much whether it's 10 files or 10,000...it's still gotta haul out the whole ugly thing. Ok, well, I only use maildir format, so can't speak to TBirds performance or quirks wrt mbox...
-- ==== Once upon a time, the Internet was a friendly, neighbors-helping-neighbors small town, and no one locked their doors. Now it's like an apartment in Bed-Stuy: you need three heavy duty pick-proof locks, one of those braces that goes from the lock to the floor, and bars on the windows.... ==== Stewart Dean, Unix System Admin, Bard College, New York 12504 sdean@bard.edu voice: 845-758-7475, fax: 845-758-7035
On 12/15/2008, Stewart Dean (sdean@bard.edu) wrote:
I would like Tbird to do just what it does now, but be more robust (and maybe a little quicker) about it. Bombproof, as they say.
I'm with you there, and I've read that the new 3.0 will have a lot of IMAP improvements, but haven't tried it yet. I tend to avoid alphas, but I'll probably give the betas a whirl,if can install both and run them side by side like I could Firefox 2 and 3...
So, I'm curious - what, exactly, did you do in TBird? You selected all of the messages in the Inbox, then... ?
I marked them for Deletion (dunno what exactly that does, maybe something in that first mbox entry or in the index or....but it does NOT xfer them to another folder, just gives them a "black spot".
Ahh, ok, I always leave 'When I delete a message' set at the default 'Move it to the trash folder'... so, my experience here is non-existent and irrelevant... ;)
--
Best regards,
Charles
On Mon, 2008-12-15 at 14:34 -0500, Stewart Dean wrote:
With
a mbox format inbox, I don't know that it matters much whether it's 10 files or 10,000...it's still gotta haul out the whole ugly thing.
Aha, mbox. And pine is your last resort (as you stated in a follow up)?
formail and procmail are a last resort! ;) Frankly, if you ask me, they really are not a bad choice for post processing and surgery in mbox anyway.
guenther -- who hacked a formail-grep wrapper script ;)
--
char *t="\10pse\0r\0dtu\0.@ghno\x4e\xc8\x79\xf4\xab\x51\x8a\x10\xf4\xf4\xc4";
main(){ char h,m=h=*t++,*x=t+2*h,c,i,l=*x,s=0; for (i=0;i
Is there a simple robust IMAP client Yes: mutt. One of the reasons we use mutt not only for regular mail access, but for troubleshooting: it simply does what you tell it to do. It doesn't try to be clever or try to do what it thinks you actually wanted to do. Apart from that, it's scriptable and heavily configurable. And autoview (combined with metamail and copiousoutput) lets you almost forget people sending you html-only or M$-doc-only mails.
to replace Pine (which I *think* is no longer supported)? Alpine?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, Dec 15, 2008 at 12:45:13PM -0500, Stewart Dean wrote:
[...]
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
I switched from Pine to Mutt (quite a while ago) and to me, Mutt is like Pine but a lot faster. OK, there are some features added ;-)
Regards
- -- tomás -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFJRz/qBcgs9XrR2kYRAsGcAJ42hWRQMgEewhV7owTJdATRyo2x+gCfb83T ibP1axKWwEuQH7PiaDVP9/g= =cNZ3 -----END PGP SIGNATURE-----
On Mon, 2008-12-15 at 12:45 -0500, Stewart Dean wrote:
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
Pine and Alpine are about the only clients (besides webmails) that can open a mailbox without downloading every single message's headers at startup. So they're probably the only clients where you can quickly open a huge mailbox and start deleting messages. I wish there were more clients that worked like that.
Words by Timo Sirainen [Tue, Dec 16, 2008 at 08:17:11AM +0200]:
On Mon, 2008-12-15 at 12:45 -0500, Stewart Dean wrote:
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
Pine and Alpine are about the only clients (besides webmails) that can open a mailbox without downloading every single message's headers at startup. So they're probably the only clients where you can quickly open a huge mailbox and start deleting messages. I wish there were more clients that worked like that.
Mutt also has header caching. Just compile it with
--enable-hcache Enable header caching
-- Jose Celestino | http://japc.uncovering.org/files/japc-pgpkey.asc
"One man’s theology is another man’s belly laugh." -- Robert A. Heinlein
On Tue, 2008-12-16 at 14:26 +0000, Jose Celestino wrote:
Words by Timo Sirainen [Tue, Dec 16, 2008 at 08:17:11AM +0200]:
On Mon, 2008-12-15 at 12:45 -0500, Stewart Dean wrote:
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
Pine and Alpine are about the only clients (besides webmails) that can open a mailbox without downloading every single message's headers at startup. So they're probably the only clients where you can quickly open a huge mailbox and start deleting messages. I wish there were more clients that worked like that.
Mutt also has header caching. Just compile it with
--enable-hcache Enable header caching
But it still downloads all the mails at some point, right? So if there are 100k new messages, it downloads their headers. Whereas Pine would only download them as necessary (one or two pagefuls at a time).
Words by Timo Sirainen [Tue, Dec 16, 2008 at 04:40:01PM +0200]:
On Tue, 2008-12-16 at 14:26 +0000, Jose Celestino wrote:
Words by Timo Sirainen [Tue, Dec 16, 2008 at 08:17:11AM +0200]:
On Mon, 2008-12-15 at 12:45 -0500, Stewart Dean wrote:
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
Pine and Alpine are about the only clients (besides webmails) that can open a mailbox without downloading every single message's headers at startup. So they're probably the only clients where you can quickly open a huge mailbox and start deleting messages. I wish there were more clients that worked like that.
Mutt also has header caching. Just compile it with
--enable-hcache Enable header caching
But it still downloads all the mails at some point, right? So if there are 100k new messages, it downloads their headers. Whereas Pine would only download them as necessary (one or two pagefuls at a time).
Absolutely right. It will download them *all* at startup if not in cache, not page by page.
-- Jose Celestino | http://japc.uncovering.org/files/japc-pgpkey.asc
"One man’s theology is another man’s belly laugh." -- Robert A. Heinlein
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Mon, 15 Dec 2008, Stewart Dean wrote:
telling me there were still 135,000 messages. In the end, I had to kill the TBird profile for that account and recreate it.
Yeah, the first and only time I opened the Postmaster mailbox with Thunderbird, it nearly rendered a Dual Core2 useless. Putting strace on it revealed, that Tbird was working with its message cache endlessly. I really waited till it finished, but to delete anything was even worse. The internal cache file is not designed for that many mails, I guess.
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
:-) Yes, I never read Postmaster's mail box with Thunderbird.
Pine has been replaced by Alpine, as others pointing out and works unproblematicly with Dovecot, regardless of how many mails there are, as long as Dovecot keeps up.
Mutt works, too. Although I don't like the input chars (same old vi vs. emacs adiction), it is far better than pine when running on the Maildir natively bypassing IMAP. After training, mutt should be more powerful than pine.
Both Mutt and Pine are superior in these emergency situations, because they do not cache the mail info locally, before they prompt the user. I guess, you can use any other client doing the same.
I'm wondering if there is something we can tell users to use when Things Are Dire. GUI would be better since it removes one of the few remaining reasons for a logon server
Well, the combination of mutt and server logon will work for any desaster case IMO.
Bye,
Steffen Kaiser -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.6 (GNU/Linux)
iD8DBQFJR2kCVJMDrex4hCIRAvobAJsFjy/XRuOWQna21MBoRLQle1WsswCggdii WqLoNUvWcjZvdhjO8/ufXuo= =6kz3 -----END PGP SIGNATURE-----
On Mon, 15 Dec 2008 12:45:13 -0500 Stewart Dean sdean@bard.edu wrote:
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
I'm wondering if there is something we can tell users to use when Things Are Dire. GUI would be better since it removes one of the few remaining reasons for a logon server
GUIwise, I have been using Sylpheed for years, both personally and professionally, and I believe it to be the best GUI-type IMAP client around. It too does the header caching and other stuff mentioned but, compared with Thunderbird, it has:
. always performed "better" (i.e., faster) . never crashed (AFAICR)
Bling-wise, it's a bit poor, but it gets the job done.
Mário Barbosa
On Thu, 18 Dec 2008 22:11:25 +0000 Mário Barbosa mplbarbosa@clix.pt wrote:
On Mon, 15 Dec 2008 12:45:13 -0500 Stewart Dean sdean@bard.edu wrote:
Is there a simple robust IMAP client to replace Pine (which I *think* is no longer supported)? GUI or TTY session?
I'm wondering if there is something we can tell users to use when Things Are Dire. GUI would be better since it removes one of the few remaining reasons for a logon server
GUIwise, I have been using Sylpheed for years, both personally and professionally, and I believe it to be the best GUI-type IMAP client around. It too does the header caching and other stuff mentioned but, compared with Thunderbird, it has:
. always performed "better" (i.e., faster) . never crashed (AFAICR)
Bling-wise, it's a bit poor, but it gets the job done.
Mário Barbosa
I would recommend Claws Mail instead, it is "the succesor" of Sylpheed and is much better... :)
BTJ
--
Bjørn T Johansen
btj@havleik.no
Someone wrote: "I understand that if you play a Windows CD backwards you hear strange Satanic messages" To which someone replied: "It's even worse than that; play it forwards and it installs Windows"
participants (14)
-
Bjørn T Johansen
-
Charles Marcus
-
Ed W
-
Edgar Fuß
-
Jose Celestino
-
Karsten Bräckelmann
-
Mário Barbosa
-
Patrick Ben Koetter
-
Rick Romero
-
Steffen Kaiser
-
Stewart Dean
-
Timo Sirainen
-
tomas@tuxteam.de
-
Udo Rader