Search results for "jmap"

Re: http API for IMAP

by Benny Pedersen

Thomas Güttler via dovecot skrev den 2019-11-13 14:40: > I would love to write a progressive web app for accessing dovecot (via > IMAP) like all other webmail is using imap > But JavaScript in the browser can only use http/https. so what ? :=) hopefully you wont run webmail over http > Is there a way to access mails in dovecot via https? google jmap, with is imho work on progress in dovecot > Maybe by a third-party tool which I don't know yet.... feel free to make another imap based webmail

6 years, 11 months

Re: http API for IMAP

by Thomas Güttler

Am 14.11.19 um 14:21 schrieb Phillip Odam via dovecot: > A HTTP API for IMAP and for that matter, POP3 and SMTP is exactly what we built where I work. Did you build upon JMAP? If not, why not? Regards, Thomas Güttler -- Thomas Guettler http://www.thomas-guettler.de/ I am looking for feedback: https://github.com/guettli/programming-guidelines

6 years, 11 months

Updated Roadmap for Dovecot?

by Tanstaafl

Hello, Just wanted to go take a peek at the latest Roadmap for dovecot, and note that the current wiki page is flagged as obsolete: https://wiki.dovecot.org/Roadmap Is there an updated version somewhere? I'm interested in the list of new features being worked on - especially JMAP support - and a rough idea of when said features might be expected to make it into a release. Thanks!

6 years, 8 months

Re: http API for IMAP

by Benny Pedersen

Ralph Seichter via dovecot skrev den 2019-11-13 17:21: > * Thomas Güttler via dovecot: > >> Is there a way to access mails in dovecot via https? > > Why on earth would that be beneficial? > > "The Internet Message Access Protocol, Version 4rev1 (IMAP4rev1) > allows a client to access and manipulate electronic mail messages > on a server." (RFC 3501) > > Putting it bluntly: Learn to use the protocol specifically designed for > the task at hand, not a protocol with different design goals that you > happen to know better. hurray for gmail.com where pop3s imaps is default disabled i see forward to have dovecot supporting jmap https://dovecot.org/pipermail/dovecot/2016-December/106491.html https://www.dovecot.nl/pipermail/dovecot/2019-May/116048.html <sorry ssl>

6 years, 11 months

Re: New Dovecot service: SMTP Submission (RFC6409)

by María Arrea

Stephan, thank you very much for your hard work. I want to ask your opinion about jmap ( http://jmap.io/ ) , do you think is a viable alternative to current IMAP + MSA ? Regards María El 12/12/17 a las 00:14, Stephan Bosch escribió: > Hi, > > As some of you know, I started implementing the SMTP submission proxy a > few years ago. It acts as a front-end for any MTA, adding the necessary > functionality for an SMTP submission service, also known as a Mail > Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409) The main > reason I created this, back then, was implementing the BURL capability > (https://tools.ietf.org/html/rfc4468) The main application of that > capability -- together with IMAP URLAUTH -- is avoiding a duplicate > upload of submitted e-mail messages; normally the message is both sent > through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, > the client can first upload the message to IMAP and then use BURL to > make the SMTP server fetch the message from IMAP for submission, thereby > avoiding a second upload. Apart from BURL, the submission proxy service > also adds the required AUTH support, avoiding the need to configure the > MTA for SASL authentication. More SMTP capabilities like CHUNKING and > SIZE are supported, without requiring the backend MTA supporting these > extensions. Other capabilities like DSN currently require support from > the backend/relay MTA. > > At this point, the submission proxy is still pretty basic. However, it > will provide a basis for adding all kinds of functionality in the (not > so distant) future. For the first time, it will be possible to act upon > message submission, rather than only message retrieval; e.g. plugins can > be devised that process outgoing messages somehow. Examples of the > things we could do are adding Sieve filtering support for outgoing > messages, or implicitly storing submitted messages to the Sent folder. > Once a plugin API is devised, you can create your own plugins. > > The reason I send this message now, is that this code is finally merged > into the Dovecot master repository. This means that it is part of the > upcoming 2.3 release. Now that it is merged, you can install and test it > from Github if you like. Feedback is of course appreciated. The > documentation is still pretty sparse, but there is currently not much to > configure. Just add "submission" to the protocols and configure the > relay MTA server. The configuration is currently only documented in the > example configuration in doc/example-config/conf.d/20-submission.conf. > The submission service is a login service, just like IMAP, POP3 and > ManageSieve, so clients are required to authenticate. The same > authentication configuration will also apply to submission, unless > you're doing protocol-specific things, in which case you may need to > amend your configuration for the new protocol. BURL support requires a > working IMAP URLAUTH implementation. > > I've updated the automated Xi Debian package builder to create an > additional dovecot-submissiond package. So, if you're using the Xi > packages, you only need to install that package and configure the relay MTA. > > Regards, > > Stephan. > > > > >

8 years, 10 months

Re: Scaling to 10 Million IMAP sessions on a single server

by KT Walrus

> On Feb 21, 2017, at 11:12 PM, Christian Balzer <chibi(a)gol.com> wrote: > > On Tue, 21 Feb 2017 09:49:39 -0500 KT Walrus wrote: > >> I just read this blog: https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-c… <https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-c…><https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-c… <https://mrotaru.wordpress.com/2013/10/10/scaling-to-12-million-concurrent-c…>> about scaling to 12 Million Concurrent Connections on a single server and it got me thinking. >> > > While that's a nice article, nothing in it was news to me or particular > complex when one does large scale stuff, like Ceph for example. > >> Would it be possible to scale Dovecot IMAP server to 10 Million IMAP sessions on a single server? >> > I'm sure Timo's answer will (or would, if he could be bothered) be along > the lines of: > "Sure, if you give me all your gold and then some for a complete rewrite > of, well, everything”. It will be a long time before I would need to scale to 10 Million users and I will be happy to pay for the rewrite of the IMAP plugin when the time comes, if not done before then by someone else. I have seen proposals for a new client protocol called JMAP that seem to be all about running a mail server at scale like an NGINX https web server can scale. That got me thinking about wether there is anything fundamental about IMAP that causes it to be difficult to scale. After looking into Dovecot’s current IMAP implementation, I think the approach was taken that fundamentally would have scaling issues (as in, one backend process per IMAP session). I see a couple years ago, work was done to “migrate” idling IMAP sessions to a single process that “remembers” the state of the IMAP session and can restore it back to a backend process when the idling is done. But, the only estimate that I have read about the “migrate idling” is that you are likely to see only a 20% reduction of the number of concurrent processes you need if you are running at 50,000 IMAP sessions per mail server. 20% reduction is not nearly enough of a benefit for scale. I would need to see at least an order of magnitude improvement to scale (and hopefully, several orders of magnitude). So, in my mind, since these IMAP sessions are long lived with infrequent bursts of activity, a better approach would be to manage the session data in memory or in an external datastore and only process using the session data when there is activity. Much like Web Sockets and even HTTPS requests are handled today for installations that need to scale to support millions of active users. As for Dovecot, I would think the work done to “migrate” idling IMAP sessions would be a good start to implementing managing a large number of sessions with a fixed pool of worker processes like other web servers do. So, my question really is: Is there anything about the IMAP protocol that would prevent an implementation from scaling to 10 Million users per server? Or, do we need to push for a new protocol like JMAP that has been designed to scale better (by being stateless with the server requests)? Kevin

9 years, 8 months

Re: Dovecot rest api

by aki.tuomi＠dovecot.fi

> On April 2, 2016 at 3:30 PM Phil Lello <phil(a)dunlop-lello.uk> wrote: > > > +1 - me too. In my case, I'm looking for a way to do webmail without imap > since the web interface wants external auth via SAML. > On 2 Apr 2016 05:54, "use experience" <useexperience(a)gmail.com> wrote: > > > Team, > > > > Need more information on dovecot rest api. Any detailed information about > > list of dovecot rest apis other than > > > > http://www.dovecot.fi/dovecot-launches-a-gmail-api-compatible-rest-api-to-e… > > will > > really helpful. > > There is a management API for Dovecot in community version these days, http://wiki2.dovecot.org/Design/DoveadmProtocol/HTTP and we are also going to provide http://jmap.io based HTTP API for accessing emails in future community release. Aki Tuomi Dovecot Oy

10 years, 7 months

RE: The future of SIS

by Laura Smith

> > Interesting, nice they use this rust, I am curious how they define this scaling. What I don't get is why are they messing with smtp. I always get a bad feeling when a company is trying to do everything. Good they are using rust and even better they've had an independent security audit (https://www.stalw.art/blog/security-audit) On the scaling side, maybe see the storage page ? (https://www.stalw.art/docs/storage/overview) The metadata is stored in a database which can be replicated. And the mails themselves can be stored in filesystem or "S3-compatible" storage, and so there are scaling options there too ? But clearly some experimentation is required to see how it works in practice. Are they messing with SMTP ? As I understand it its an IMAP/JMAP server. And (like Dovecot) it has LMTP for getting mail into it from e.g. Postfix ? From my reading of the docs it looks like SMTP is only there if you don't want to use LMTP to get mail into it ?

3 years

Re: New Dovecot service: SMTP Submission (RFC6409)

by Stephan Bosch

Op 14-12-2017 om 8:26 schreef María Arrea: > > Stephan, thank you very much for your hard work. I want to ask > your opinion about jmap ( http://jmap.io/ ) , do you think is a viable > alternative to current IMAP + MSA ? > Difficult to tell at this point, as It is not finished yet. It all depends on whether people will want to implement it. Still, technology-wise I think it is solid and, as far as I know now, we'll be implementing it at some point. Regards, Stephan. > Regards > > María > > El 12/12/17 a las 00:14, Stephan Bosch escribió: >> Hi, >> >> As some of you know, I started implementing the SMTP submission proxy a >> few years ago. It acts as a front-end for any MTA, adding the necessary >> functionality for an SMTP submission service, also known as a Mail >> Submission Agent (MSA) (https://tools.ietf.org/html/rfc6409) The main >> reason I created this, back then, was implementing the BURL capability >> (https://tools.ietf.org/html/rfc4468) The main application of that >> capability -- together with IMAP URLAUTH -- is avoiding a duplicate >> upload of submitted e-mail messages; normally the message is both sent >> through SMTP and uploaded to the "Sent" folder through IMAP. Using BURL, >> the client can first upload the message to IMAP and then use BURL to >> make the SMTP server fetch the message from IMAP for submission, thereby >> avoiding a second upload. Apart from BURL, the submission proxy service >> also adds the required AUTH support, avoiding the need to configure the >> MTA for SASL authentication. More SMTP capabilities like CHUNKING and >> SIZE are supported, without requiring the backend MTA supporting these >> extensions. Other capabilities like DSN currently require support from >> the backend/relay MTA. >> >> At this point, the submission proxy is still pretty basic. However, it >> will provide a basis for adding all kinds of functionality in the (not >> so distant) future. For the first time, it will be possible to act upon >> message submission, rather than only message retrieval; e.g. plugins can >> be devised that process outgoing messages somehow. Examples of the >> things we could do are adding Sieve filtering support for outgoing >> messages, or implicitly storing submitted messages to the Sent folder. >> Once a plugin API is devised, you can create your own plugins. >> >> The reason I send this message now, is that this code is finally merged >> into the Dovecot master repository. This means that it is part of the >> upcoming 2.3 release. Now that it is merged, you can install and test it >> from Github if you like. Feedback is of course appreciated. The >> documentation is still pretty sparse, but there is currently not much to >> configure. Just add "submission" to the protocols and configure the >> relay MTA server. The configuration is currently only documented in the >> example configuration in doc/example-config/conf.d/20-submission.conf. >> The submission service is a login service, just like IMAP, POP3 and >> ManageSieve, so clients are required to authenticate. The same >> authentication configuration will also apply to submission, unless >> you're doing protocol-specific things, in which case you may need to >> amend your configuration for the new protocol. BURL support requires a >> working IMAP URLAUTH implementation. >> >> I've updated the automated Xi Debian package builder to create an >> additional dovecot-submissiond package. So, if you're using the Xi >> packages, you only need to install that package and configure the >> relay MTA. >> >> Regards, >> >> Stephan. >> >> >> >> >> >

8 years, 10 months

Re: Scaling to 10 Million IMAP sessions on a single server

by Timo Sirainen

On 22 Feb 2017, at 17.07, KT Walrus <kevin(a)my.walr.us> wrote: > > I have seen proposals for a new client protocol called JMAP that seem to be all about running a mail server at scale like an NGINX https web server can scale. That got me thinking about wether there is anything fundamental about IMAP that causes it to be difficult to scale. After looking into Dovecot’s current IMAP implementation, I think the approach was taken that fundamentally would have scaling issues (as in, one backend process per IMAP session). I see a couple years ago, work was done to “migrate” idling IMAP sessions to a single process that “remembers” the state of the IMAP session and can restore it back to a backend process when the idling is done. > > But, the only estimate that I have read about the “migrate idling” is that you are likely to see only a 20% reduction of the number of concurrent processes you need if you are running at 50,000 IMAP sessions per mail server. 20% reduction is not nearly enough of a benefit for scale. I would need to see at least an order of magnitude improvement to scale (and hopefully, several orders of magnitude). My long-term plans are something like this: * imap-hibernate process can be used more aggressively. Not necessarily even for just IDLEing sessions, but for any session that isn't actively being used. And actually if the server is too busy, even active sessions could be hibernated. That would be somewhat similar to cooperative multitasking. When this is done, you can think of the current imap processes as the worker processes. * More state will be transferred to imap-hibernate process, so it can perform simpler commands without recreating the IMAP process. For example STATUS replies can be returned from cached state as long as it hasn't actually changed. * imap-hibernate is currently tracking changed state via inotify (etc.) This mostly work, but it's also unnecessarily sometimes waking up. For example just because one IMAP session performed a FETCH that added something to dovecot.index.cache, it doesn't mean that there are any real changes. We'll need some mail plugin that notifies imap-hibernate process when some real change has happened. * Hibernated sessions can even be moved away entirely from backends into IMAP proxies. The IMAP proxy can then reconnect to backend to re-establish the session. This allows even switching backends entirely, as long as the storage is shared. This requires that backends notify the proxy whenever something changes to the user, which is mostly a continuation of the previous item (just TCP notification instead of UNIX socket notification). * IMAP proxies can also perform similar limited functionality as imap-hibernate processes. Possibly running the same imap-hibernate processes. * And kind of a reverse of hibernation: imap processes can also preserve the user's imap session and opened folder indexes in memory even after the IMAP client has disconnected. If the same user connects back, the imap process can quickly be re-used with all the state already open. This is especially useful for client that create many short-lived connections, such as webmails. So after all these changes there would practically be something like 1000 imap processes constantly open and either doing work or waiting for a recently disconnected IMAP client to come back. As Christian already mentioned, the Dovecot proxies are supposed to be able to handle quite a lot of connections. I wouldn't be surprised if you can already do millions of connections with them. Most of our customers haven't tried scaling them very hard because they don't really want to create multiple IP addresses for servers, which is required to avoid running out of TCP ports (or I guess there could be multiple destination ports, but that also complicates things and Dovecot doesn't currently support that in an easy way either). > Is there anything about the IMAP protocol that would prevent an implementation from scaling to 10 Million users per server? Or, do we need to push for a new protocol like JMAP that has been designed to scale better (by being stateless with the server requests)? I guess mainly the message sequence numbers in IMAP protocol makes this more difficult, but it's not an impossible problem to solve.

9 years, 8 months

Search results for query "jmap"