Announce: Plugin for iOS Push Email support
I’ve been hacking on a personal side project to support native iOS Push Email in Dovecot. This is specifically for people who are migrating their mail away from OS X Server while keeping their existing Push Email functionality.
Native Push Email has some great advantages: it speeds up email notifications (usually within seconds of being handled by dovecot-lda) and it improves battery life since the native notifications flow over a single highly optimized connection to Apple’s infrastructure.
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested.
Please note that it is not possible to use this project without legally running a copy
of OS X Server. You can purchase OS X Server on the Mac App Store or download
it for free if you are a registered Mac or iOS developer.This feature is enabled by two projects:
https://github.com/st3fan/dovecot-xaps-plugin https://github.com/st3fan/dovecot-xaps-plugin https://github.com/st3fan/dovecot-xaps-daemon https://github.com/st3fan/dovecot-xaps-daemon
Both projects contain a README that describes how to get things going. I have only developed and tested on Ubuntu 12.04.5 so ideally you run the same version if you are interested in playing with this project.
You will need to compile some code and not be afraid of a little admin work.
I am also interested in a code review. The Dovecot plugin API is barely documented and I had to guess a lot of things by looking at other plugins. An extra pair of eyes specifically on that code would be awesome. Also from a security perspective.
Please file bugs!
S.
This is interesting; I'm not sure if I got that right from scanning the READMEs. Do I really need to have a working OSX-server setup which I then do migrate away from? What makes this certificate that special such that I only can export it from the existing server setup?
-M
Am 05.09.2014 um 03:35 schrieb Stefan Arentz stefan@arentz.ca:
I’ve been hacking on a personal side project to support native iOS Push Email in Dovecot. This is specifically for people who are migrating their mail away from OS X Server while keeping their existing Push Email functionality.
Native Push Email has some great advantages: it speeds up email notifications (usually within seconds of being handled by dovecot-lda) and it improves battery life since the native notifications flow over a single highly optimized connection to Apple’s infrastructure.
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested.
Please note that it is not possible to use this project without legally running a copy of OS X Server. You can purchase OS X Server on the Mac App Store or download it for free if you are a registered Mac or iOS developer.
This feature is enabled by two projects:
https://github.com/st3fan/dovecot-xaps-plugin https://github.com/st3fan/dovecot-xaps-plugin https://github.com/st3fan/dovecot-xaps-daemon https://github.com/st3fan/dovecot-xaps-daemon
Both projects contain a README that describes how to get things going. I have only developed and tested on Ubuntu 12.04.5 so ideally you run the same version if you are interested in playing with this project.
You will need to compile some code and not be afraid of a little admin work.
I am also interested in a code review. The Dovecot plugin API is barely documented and I had to guess a lot of things by looking at other plugins. An extra pair of eyes specifically on that code would be awesome. Also from a security perspective.
Please file bugs!
S.
On 9/4/2014 9:35 PM, Stefan Arentz stefan@arentz.ca wrote:
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested.
Please note that it is not possible to use this project without legally running a copy of OS X Server. You can purchase OS X Server on the Mac App Store or download it for free if you are a registered Mac or iOS developer.
Well, that is pretty much a deal breaker for just about everyone (that isn't already running an OSX server)...
Would you mind explaining *why* the above is required? Is this just a temporary limitation of some kind?
On Sep 5, 2014, at 5:53 AM, Charles Marcus CMarcus@Media-Brokers.com wrote:
On 9/4/2014 9:35 PM, Stefan Arentz stefan@arentz.ca wrote:
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested.
Please note that it is not possible to use this project without legally running a copy of OS X Server. You can purchase OS X Server on the Mac App Store or download it for free if you are a registered Mac or iOS developer.
Well, that is pretty much a deal breaker for just about everyone (that isn't already running an OSX server)...
Would you mind explaining *why* the above is required? Is this just a temporary limitation of some kind?
Sorry I should have been more clear.
Let me explain.
To send native push email notifications, you need a certificate that is generated by Apple.
When you enable Push Email on OS X Server, that certificate will be generated for you and stored in your Keychain.
There is no other way to generate this certificate. This is why you need a legal copy of OS X Server.
You don’t actually have to run OS X Server. I migrated away from OS X Server to regular Dovecot on Ubuntu. So I simply took the generated certificate with me. How to export and convert it is described in the README of the daemon project.
This is possible because the certificate is not bound to specific hardware or a specific email domain. It is however connected to your Apple ID.
I understand that this is not ideal but unfortunately there is no way around this. OS X Server however is just $19.99.
When my code is a bit more final, I will try to contact Apple and find out if they are willing to open up the certificate generation. That would sure benefit a larger audience.
S.
On 9/5/2014 9:22 AM, Stefan Arentz stefan@arentz.ca wrote:
When you enable Push Email on OS X Server, that certificate will be generated for you and stored in your Keychain.
There is no other way to generate this certificate. This is why you need a legal copy of OS X Server.
You don’t actually have to run OS X Server. I migrated away from OS X Server to regular Dovecot on Ubuntu. So I simply took the generated certificate with me. How to export and convert it is described in the README of the daemon project.
This is possible because the certificate is not bound to specific hardware or a specific email domain. It is however connected to your Apple ID.
I understand that this is not ideal but unfortunately there is no way around this. OS X Server however is just $19.99.
When my code is a bit more final, I will try to contact Apple and find out if they are willing to open up the certificate generation. That would sure benefit a larger audience.
Ok, thanks for the explanation...
Gotta love (hate) closed source software...
"... I will try to contact Apple and find out if they are willing to open up the certificate generation. "
I'm willing to bet the answer will be no.
Apple != Open On Sep 5, 2014 9:22 AM, "Stefan Arentz" stefan@arentz.ca wrote:
On Sep 5, 2014, at 5:53 AM, Charles Marcus CMarcus@Media-Brokers.com wrote:
On 9/4/2014 9:35 PM, Stefan Arentz stefan@arentz.ca wrote:
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested.
Please note that it is not possible to use this project withoutlegally running a copy of OS X Server. You can purchase OS X Server on the Mac App Store or download it for free if you are a registered Mac or iOS developer.
Well, that is pretty much a deal breaker for just about everyone (that isn't already running an OSX server)...
Would you mind explaining *why* the above is required? Is this just a temporary limitation of some kind?
Sorry I should have been more clear.
Let me explain.
To send native push email notifications, you need a certificate that is generated by Apple.
When you enable Push Email on OS X Server, that certificate will be generated for you and stored in your Keychain.
There is no other way to generate this certificate. This is why you need a legal copy of OS X Server.
You don’t actually have to run OS X Server. I migrated away from OS X Server to regular Dovecot on Ubuntu. So I simply took the generated certificate with me. How to export and convert it is described in the README of the daemon project.
This is possible because the certificate is not bound to specific hardware or a specific email domain. It is however connected to your Apple ID.
I understand that this is not ideal but unfortunately there is no way around this. OS X Server however is just $19.99.
When my code is a bit more final, I will try to contact Apple and find out if they are willing to open up the certificate generation. That would sure benefit a larger audience.
S.
No matter what the answer's gonna be the "solution" till then still might be the z-push+exchange-approach? Or did I miss anything more sophisticated out there?
Am 05.09.2014 um 15:22 schrieb Stefan Arentz stefan@arentz.ca:
On Sep 5, 2014, at 5:53 AM, Charles Marcus CMarcus@Media-Brokers.com wrote:
On 9/4/2014 9:35 PM, Stefan Arentz stefan@arentz.ca wrote:
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested.
Please note that it is not possible to use this project without legally running a copy of OS X Server. You can purchase OS X Server on the Mac App Store or download it for free if you are a registered Mac or iOS developer.
Well, that is pretty much a deal breaker for just about everyone (that isn't already running an OSX server)...
Would you mind explaining *why* the above is required? Is this just a temporary limitation of some kind?
Sorry I should have been more clear.
Let me explain.
To send native push email notifications, you need a certificate that is generated by Apple.
When you enable Push Email on OS X Server, that certificate will be generated for you and stored in your Keychain.
There is no other way to generate this certificate. This is why you need a legal copy of OS X Server.
You don’t actually have to run OS X Server. I migrated away from OS X Server to regular Dovecot on Ubuntu. So I simply took the generated certificate with me. How to export and convert it is described in the README of the daemon project.
This is possible because the certificate is not bound to specific hardware or a specific email domain. It is however connected to your Apple ID.
I understand that this is not ideal but unfortunately there is no way around this. OS X Server however is just $19.99.
When my code is a bit more final, I will try to contact Apple and find out if they are willing to open up the certificate generation. That would sure benefit a larger audience.
S.
Il 05/09/2014 03:35, Stefan Arentz ha scritto:
I’ve been hacking on a personal side project to support native iOS Push Email in Dovecot. This is specifically for people who are migrating their mail away from OS X Server while keeping their existing Push Email functionality.
Native Push Email has some great advantages: it speeds up email notifications (usually within seconds of being handled by dovecot-lda) and it improves battery life since the native notifications flow over a single highly optimized connection to Apple’s infrastructure.
Although this is at version 0.1, it is working pretty well for me and I am looking for some additional testers that are interested. Hi Stefan,
I have installed your code on my personal email server (Dovecot 2.2.13 and Debian 7.6) and works fine! The only note is to copy the plugins file (imap_xaps_plugin.so and xaps_plugin.so) manually in /usr/lib/dovecot/.
I hope you will release a version with the ability to run in background the daemon.
Thanks
On 05 Sep 2014, at 04:35, Stefan Arentz stefan@arentz.ca wrote:
I’ve been hacking on a personal side project to support native iOS Push Email in Dovecot. This is specifically for people who are migrating their mail away from OS X Server while keeping their existing Push Email functionality.
Nice. I had been planning to look into this as well, but never had much time. Some comments from a quick look:
xaps_str_append_quoted() doesn't quote CR/LF/TABs. Of course those shouldn't normally exist in any of the fields, but older Dovecot versions would have allowed creating folder names with them. Using the str_append_tabescaped() would escape them as well, although then you'd have to implement the unescaping also with Python.
ideally the communication between Dovecot and xaps-daemon would be done asynchronously, but as a quick change you could also use i_stream_create_fd() + i_stream_read_next_line() instead of the strtok_r call. Oh and in internal Dovecot protocols it has always used LF, not CRLF, which would also make it a bit easier to handle.
in xaps_notify() especially async IO would be much better to avoid latency. It's also easier to do than in init, since it wouldn't really even need to wait for the response. Even without async IO does it really even need to read the answer from the server?
also to avoid bursts of notifications in some situations, you could use timeout_add(NOTIFY_DELAY_MSECS, ...) and have the timeout function send the notification. The delay being for example 0,5 seconds or maybe configurable.
BTW. We could have some other Dovecot plugins that need to be developed if you want more (paid) work. :) We need to hire more developers to reduce my work load..
On Sep 15, 2014, at 8:58 AM, Timo Sirainen tss@iki.fi wrote:
On 05 Sep 2014, at 04:35, Stefan Arentz stefan@arentz.ca wrote:
I’ve been hacking on a personal side project to support native iOS Push Email in Dovecot. This is specifically for people who are migrating their mail away from OS X Server while keeping their existing Push Email functionality.
Nice. I had been planning to look into this as well, but never had much time. Some comments from a quick look:
xaps_str_append_quoted() doesn't quote CR/LF/TABs. Of course those shouldn't normally exist in any of the fields, but older Dovecot versions would have allowed creating folder names with them. Using the str_append_tabescaped() would escape them as well, although then you'd have to implement the unescaping also with Python.
ideally the communication between Dovecot and xaps-daemon would be done asynchronously, but as a quick change you could also use i_stream_create_fd() + i_stream_read_next_line() instead of the strtok_r call. Oh and in internal Dovecot protocols it has always used LF, not CRLF, which would also make it a bit easier to handle.
in xaps_notify() especially async IO would be much better to avoid latency. It's also easier to do than in init, since it wouldn't really even need to wait for the response. Even without async IO does it really even need to read the answer from the server?
also to avoid bursts of notifications in some situations, you could use timeout_add(NOTIFY_DELAY_MSECS, ...) and have the timeout function send the notification. The delay being for example 0,5 seconds or maybe configurable.
Hi Timo,
Thank you for these hints. I appreciate your input. This all sounds pretty good and I will try to make these changes soon and push out a new release.
For people who are interested in trying our this project, I have made considerable improvements to the daemon side of things. It is now a proper Twisted application that you can start using twistd, so that it goes in the background, writes a pid file, etc.
See https://github.com/st3fan/dovecot-xaps-daemon/blob/master/README.md#installi...
S.
What would happen if say, the dovegot xaps plugin is installed and the
daemon crashed?
Would peoples iphones just wait for the push notification? get email
every like 30/60min? or would it just use imap idle instead?
Just wondering about sideeffects :)
Quoting Stefan Arentz stefan@arentz.ca:
On Sep 15, 2014, at 8:58 AM, Timo Sirainen tss@iki.fi wrote:
On 05 Sep 2014, at 04:35, Stefan Arentz stefan@arentz.ca wrote:
I’ve been hacking on a personal side project to support native iOS
Push Email in Dovecot. This is specifically for people who are
migrating their mail away from OS X Server while keeping their
existing Push Email functionality.Nice. I had been planning to look into this as well, but never had
much time. Some comments from a quick look:
xaps_str_append_quoted() doesn't quote CR/LF/TABs. Of course
those shouldn't normally exist in any of the fields, but older
Dovecot versions would have allowed creating folder names with
them. Using the str_append_tabescaped() would escape them as well,
although then you'd have to implement the unescaping also with
Python.ideally the communication between Dovecot and xaps-daemon would
be done asynchronously, but as a quick change you could also use
i_stream_create_fd() + i_stream_read_next_line() instead of the
strtok_r call. Oh and in internal Dovecot protocols it has always
used LF, not CRLF, which would also make it a bit easier to handle.in xaps_notify() especially async IO would be much better to
avoid latency. It's also easier to do than in init, since it
wouldn't really even need to wait for the response. Even without
async IO does it really even need to read the answer from the server?also to avoid bursts of notifications in some situations, you
could use timeout_add(NOTIFY_DELAY_MSECS, ...) and have the timeout
function send the notification. The delay being for example 0,5
seconds or maybe configurable.Hi Timo,
Thank you for these hints. I appreciate your input. This all sounds
pretty good and I will try to make these changes soon and push out a
new release.For people who are interested in trying our this project, I have
made considerable improvements to the daemon side of things. It is
now a proper Twisted application that you can start using twistd, so
that it goes in the background, writes a pid file, etc.See
https://github.com/st3fan/dovecot-xaps-daemon/blob/master/README.md#installi...S.
participants (7)
-
admin
-
Alessio Cecchi
-
Charles Marcus
-
Daniel Reynolds
-
Patrick Domack
-
Stefan Arentz
-
Timo Sirainen