[Dovecot] dovecot as gmail imap proxy
hi, may be this is a bit of topic, but i hope not. it's always a big question which is the better to choose a hosting provider to keep and manage you mail or setup your own mail server, hosting, virus and spam filter etc.. now as more and more people and company move to gmail or google apps mail service it seems google can't handle the load and it's getting more and more slower. what's more not just the imap interface but nowadays the web interface used to be hang or even stop working. it's getting more and more annoying that the speed is worst than in a 10 years ago. i thing about how can this be solved without totally give up gmail as a mail service provider (and may be they can solve it in a few months/years:-). would it be possible to create a dovecot server as an imap proxy for gmail and google apps? imho many people/company would be love to use such a setup where he can use gmail in vacation or any other time, but when the email is real critical apps (not acceptable to wait 1 minutes for an email to load or to open a folder/label) than he can use a proxy server eg. dovecot which act as a imap proxy to gmail and can be hosted on closer, faster server and this speed problem never happend. what's more it can be used as a mail backup server too. anybody has any such setup, idea? would it be possible with dovecot. any other suggestion? yours.
-- Levente "Si vis pacem para bellum!"
To have a speed increase, dovecot needs direct access to the emails. So you'll need them stored locally. You can't use it as a proxy, just to speed up things, because dovecot would just need to wait for gmail too, to load an email (except if dovecot would cache the emails, but then again, it's just a cache, and you're talking about 'critical applications').
I'm not sure, but I don't think there's even a way to run dovecot as a proxy.
Why don't you forward mail from your gmail account to your own server? That's so much easier and more common. You get the benefit of good filtering, and you have your own server to access your mail. If your own server goes down, you still have gmail as backup.
Alexander
Farkas Levente wrote:
hi, may be this is a bit of topic, but i hope not. it's always a big question which is the better to choose a hosting provider to keep and manage you mail or setup your own mail server, hosting, virus and spam filter etc.. now as more and more people and company move to gmail or google apps mail service it seems google can't handle the load and it's getting more and more slower. what's more not just the imap interface but nowadays the web interface used to be hang or even stop working. it's getting more and more annoying that the speed is worst than in a 10 years ago. i thing about how can this be solved without totally give up gmail as a mail service provider (and may be they can solve it in a few months/years:-). would it be possible to create a dovecot server as an imap proxy for gmail and google apps? imho many people/company would be love to use such a setup where he can use gmail in vacation or any other time, but when the email is real critical apps (not acceptable to wait 1 minutes for an email to load or to open a folder/label) than he can use a proxy server eg. dovecot which act as a imap proxy to gmail and can be hosted on closer, faster server and this speed problem never happend. what's more it can be used as a mail backup server too. anybody has any such setup, idea? would it be possible with dovecot. any other suggestion? yours.
I do this exact thing, and it is faster than gmail itself. With IDLE, I get the msg alert on my dovecot server before the web interface shows a new message.
Also, far more reliable, with a superior UI.
On 2008 Jun 17 (Tue) at 17:13:32 +0200 (+0200), Alexander Prinsier wrote: :Why don't you forward mail from your gmail account to your own server? :That's so much easier and more common. You get the benefit of good :filtering, and you have your own server to access your mail. If your own :server goes down, you still have gmail as backup.
-- New Hampshire law forbids you to tap your feet, nod your head, or in any way keep time to the music in a tavern, restaurant, or cafe.
Peter Hessler wrote:
I do this exact thing, and it is faster than gmail itself. With IDLE, I get the msg alert on my dovecot server before the web interface shows a new message.
which exact thing? and what does the "with idle" means?
Also, far more reliable, with a superior UI.
On 2008 Jun 17 (Tue) at 17:13:32 +0200 (+0200), Alexander Prinsier wrote: :Why don't you forward mail from your gmail account to your own server? :That's so much easier and more common. You get the benefit of good :filtering, and you have your own server to access your mail. If your own :server goes down, you still have gmail as backup.
-- Levente "Si vis pacem para bellum!"
On Tue, 17 Jun 2008, Peter Hessler wrote:
I do this exact thing, and it is faster than gmail itself. With IDLE, I get the msg alert on my dovecot server before the web interface shows a new message.
Also, far more reliable, with a superior UI.
Which is the superior UI? Honest question - I'm shopping around for nice IMAP interfaces, if that's what you mean. I'd love to know what you're using.
-- Asheesh.
-- Marriage is the triumph of imagination over intelligence. Second marriage is the triumph of hope over experience.
On 2008 Jun 17 (Tue) at 17:18:20 -0400 (-0400), Asheesh Laroia wrote:
On Tue, 17 Jun 2008, Peter Hessler wrote:
I do this exact thing, and it is faster than gmail itself. With IDLE, I get the msg alert on my dovecot server before the web interface shows a new message.
Also, far more reliable, with a superior UI.
Which is the superior UI? Honest question - I'm shopping around for nice IMAP interfaces, if that's what you mean. I'd love to know what you're
using.-- Asheesh.
-- Marriage is the triumph of imagination over intelligence. Second marriage is the triumph of hope over experience.
I <3 mutt. Easy enough, but lots of power is available.
I haven't found any webmail interfaces that I like, but then again, I have yet to find *any* web interface that I like.
-- Marriage is the only adventure open to the cowardly. -- Voltaire
On Tue, Jun 17, 2008 at 03:37:30PM -0700, Peter Hessler wrote:
I <3 mutt. Easy enough, but lots of power is available.
Yes, mutt is great! :-)
Btw, you can use mutt with dovecot without running dovecot as a daemon/server, by putting this in your .muttrc:
set tunnel="MAIL=/home/geert/mbox /usr/pkg/libexec/dovecot/imap"
(adapt paths for your own system)
This way mutt will talk to the dovecot imap binary via stdin/stdout. This will also work over ssh. This way you don't need to run dovecot as a server and you still benefit from eg. the cache etc.
GeertOn Wed, 2008-06-18 at 10:32 +0200, Geert Hendrickx wrote:
On Tue, Jun 17, 2008 at 03:37:30PM -0700, Peter Hessler wrote:
I <3 mutt. Easy enough, but lots of power is available.
Yes, mutt is great! :-)
Btw, you can use mutt with dovecot without running dovecot as a daemon/server, by putting this in your .muttrc:
set tunnel="MAIL=/home/geert/mbox /usr/pkg/libexec/dovecot/imap"
"dovecot --exec-mail imap" is better. Then it uses the settings from dovecot.conf as well as the proper default settings. With the above you're for example using mbox_lazy_writes=no
On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
would it be possible to create a dovecot server as an imap proxy for
gmail and google apps? imho many people/company would be love to use such a setup where he
can use gmail in vacation or any other time, but when the email is
real critical apps (not acceptable to wait 1 minutes for an email to
load or to open a folder/label) than he can use a proxy server eg.
dovecot which act as a imap proxy to gmail and can be hosted on
closer, faster server and this speed problem never happend. what's
more it can be used as a mail backup server too.
I've thought about a caching IMAP mail storage backend for Dovecot and
even started implementing it once, but it didn't get very far. If
someone wants to try to continue it, I can send what I've written so
far. Although it's mostly a copy of cydir backend with a tiny bit of
it replaced with IMAP code.
On Jun 17, 2008, at 6:14 PM, Timo Sirainen wrote:
On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
would it be possible to create a dovecot server as an imap proxy
for gmail and google apps? imho many people/company would be love to use such a setup where he
can use gmail in vacation or any other time, but when the email is
real critical apps (not acceptable to wait 1 minutes for an email
to load or to open a folder/label) than he can use a proxy server
eg. dovecot which act as a imap proxy to gmail and can be hosted on
closer, faster server and this speed problem never happend. what's
more it can be used as a mail backup server too.I've thought about a caching IMAP mail storage backend for Dovecot
and even started implementing it once, but it didn't get very far.
If someone wants to try to continue it, I can send what I've written
so far. Although it's mostly a copy of cydir backend with a tiny bit
of it replaced with IMAP code.
Now that I think of it, the IMAP backend should do only direct IMAP
backend accessing and if caching is wanted it could be a generic
middle layer. Might be also useful for other things like caching mails
from NFS server. Although Linux already has a cachefs for that.
Timo Sirainen wrote:
On Jun 17, 2008, at 6:14 PM, Timo Sirainen wrote:
On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
would it be possible to create a dovecot server as an imap proxy for gmail and google apps? imho many people/company would be love to use such a setup where he can use gmail in vacation or any other time, but when the email is real critical apps (not acceptable to wait 1 minutes for an email to load or to open a folder/label) than he can use a proxy server eg. dovecot which act as a imap proxy to gmail and can be hosted on closer, faster server and this speed problem never happend. what's more it can be used as a mail backup server too.
I've thought about a caching IMAP mail storage backend for Dovecot and even started implementing it once, but it didn't get very far. If someone wants to try to continue it, I can send what I've written so far. Although it's mostly a copy of cydir backend with a tiny bit of it replaced with IMAP code.
Now that I think of it, the IMAP backend should do only direct IMAP backend accessing and if caching is wanted it could be a generic middle layer. Might be also useful for other things like caching mails from NFS server. Although Linux already has a cachefs for that.
yes, that's exactly what i'm thinking about. all imap command which issued by the client to the middle layer should have to propagate back to the backed (in this case gmail). but imho the hard part that we've to assume that the backend (in this case gmail) is slow, so the middle layer have to cache the imap commands to be able to propagate all command to the backend (and command queue for all mailbox). so if the connection between dovecot and gmail are fast then both has the same messages and flags, etc. if the connection is slow than gmail may have a small delay, but this delay probably won't be visible to the user (if he not use the web interface at the same time too).
-- Levente "Si vis pacem para bellum!"
Timo Sirainen wrote:
On Jun 17, 2008, at 6:14 PM, Timo Sirainen wrote:
On Jun 17, 2008, at 6:03 PM, Farkas Levente wrote:
would it be possible to create a dovecot server as an imap proxy for gmail and google apps? imho many people/company would be love to use such a setup where he can use gmail in vacation or any other time, but when the email is real critical apps (not acceptable to wait 1 minutes for an email to load or to open a folder/label) than he can use a proxy server eg. dovecot which act as a imap proxy to gmail and can be hosted on closer, faster server and this speed problem never happend. what's more it can be used as a mail backup server too.
I've thought about a caching IMAP mail storage backend for Dovecot and even started implementing it once, but it didn't get very far. If someone wants to try to continue it, I can send what I've written so far. Although it's mostly a copy of cydir backend with a tiny bit of it replaced with IMAP code.
Now that I think of it, the IMAP backend should do only direct IMAP backend accessing and if caching is wanted it could be a generic middle layer. Might be also useful for other things like caching mails from NFS server. Although Linux already has a cachefs for that.
it seems other people also thing about this: http://community.livejournal.com/linux/1748995.html so it'd be useful thing for many of us:-)
-- Levente "Si vis pacem para bellum!"
On Tue, 17 Jun 2008, Farkas Levente wrote:
hi, may be this is a bit of topic, but i hope not. it's always a big question which is the better to choose a hosting provider to keep and manage you mail or setup your own mail server, hosting, virus and spam filter etc.. now as more and more people and company move to gmail or google apps mail service it seems google can't handle the load and it's getting more and more slower. what's more not just the imap interface but nowadays the web interface used to be hang or even stop working. it's getting more and more annoying that the speed is worst than in a 10 years ago. i thing about how can this be solved without totally give up gmail as a mail service provider (and may be they can solve it in a few months/years:-). would it be possible to create a dovecot server as an imap proxy for gmail and google apps?
What I have is:
- rose, my primary server (but far away)
- supercore, another server at my parents' house
rose is the primary MX.
I run offlineimap on supercore to sync a local Maildir on supercore with rose's IMAP - that way, my parents can use a server with <1ms ping, and as they use Dovecot to modify the local Maildir on supercore, a minute later offlineimap will wake up and create IMAP operations to synchronize rose with the state on supercore.
FWIW.
-- Asheesh.
-- Real Users find the one combination of bizarre input values that shuts down the system for days.
Asheesh Laroia wrote:
On Tue, 17 Jun 2008, Farkas Levente wrote:
hi, may be this is a bit of topic, but i hope not. it's always a big question which is the better to choose a hosting provider to keep and manage you mail or setup your own mail server, hosting, virus and spam filter etc.. now as more and more people and company move to gmail or google apps mail service it seems google can't handle the load and it's getting more and more slower. what's more not just the imap interface but nowadays the web interface used to be hang or even stop working. it's getting more and more annoying that the speed is worst than in a 10 years ago. i thing about how can this be solved without totally give up gmail as a mail service provider (and may be they can solve it in a few months/years:-). would it be possible to create a dovecot server as an imap proxy for gmail and google apps?
What I have is:
- rose, my primary server (but far away)
- supercore, another server at my parents' house
rose is the primary MX.
I run offlineimap on supercore to sync a local Maildir on supercore with rose's IMAP - that way, my parents can use a server with <1ms ping, and as they use Dovecot to modify the local Maildir on supercore, a minute later offlineimap will wake up and create IMAP operations to synchronize rose with the state on supercore.
i'm just look into offlineimap. half of the code for ui, another part to implement maildir and imap. so it seems for me there is only a little code which do the synchronization. Timo what do you think how much work would be add such a middleware imap server capabilities (ie. imap backend) to dovecot?
-- Levente "Si vis pacem para bellum!"
On Jun 18, 2008, at 2:59 PM, Farkas Levente wrote:
Timo what do you think how much work would be add such a middleware
imap server capabilities (ie. imap backend) to dovecot?
I think it would be quite a lot of work. Definitely a lot more than I
have time for currently. The most difficult thing is probably writing
IMAP parsing and event handling for it. I've written such code for my
imaptest tool already though, maybe at some point they could be more
or less merged to a common lib-imap-client.
participants (6)
-
Alexander Prinsier
-
Asheesh Laroia
-
Farkas Levente
-
Geert Hendrickx
-
Peter Hessler
-
Timo Sirainen