My advice for anyone who wants to be able to keep and search very large monolithic mailstores would be: synchronise (e.g. via https://isync.sourceforge.io/ or fetchmail or getmail or rsync or Unison or whatever) those mailstores from the server onto your local filesystem; use maildir on your local filesystem; and use either Mutt's "limiting" functions, or notmuch's index/search functions, for searching/browsing.
Wouldn't it be very tedious and time consuming to sync mailboxes from the server onto our local filesystems?
Regards,
Mr. Turritopsis Dohrnii Teo En Ming Targeted Individual in Singapore 20 Apr 2022 Wednesday
On Wed, 20 Apr 2022 at 14:21, Sam Kuper sampablokuper@posteo.net wrote:
On Tue, Apr 19, 2022 at 07:26:10PM -0600, Shawn Heisey wrote:
I would bet that if you accessed a gmail folder with 5 million messages in it using IMAP, you would have similar problems with it to those that have been described here in this thread. IMAP is a beautiful protocol, but I don't think it was designed for handling that many messages.
This. Sadly, Mark Crispin (author of IMAP and IMAP2) is no longer with us to confirm.
Even at just ~100B for each message's headers, your IMAP2 client would likely need at least ~500MB free RAM to load 5 million mails.
By the time Mark stopped working on UW IMAP (the reference IMAP implementation, aka Panda IMAP), c.2010, even top-of-the-range smartphones typically had only ~512MB RAM total, and top-of-the-range ThinkPads had max ~4GiB (which was the upper limit of what 32-bit operating systems, still popular then, could handle).
When IMAP2 was invented, c.1988-1990, RAM like that was basically supercomputer territory.
Had Mark intended or expected IMAP2 users to have had supercomputers at their disposal, and to be searching such large volumes of mail over the protocol, I suspect he would have designed the protocol differently: for raw efficiency over human readability.
My advice for anyone who wants to be able to keep and search very large monolithic mailstores would be: synchronise (e.g. via https://isync.sourceforge.io/ or fetchmail or getmail or rsync or Unison or whatever) those mailstores from the server onto your local filesystem; use maildir on your local filesystem; and use either Mutt's "limiting" functions, or notmuch's index/search functions, for searching/browsing.
Good luck in your quest!
Sam