macOS Notes.app IMAP Syncing Not Working
I’m requesting help getting macOS’s Notes.app to sync notes with a new dovecot IMAP server.
Thanks to dovecot’s excellent online documentation and help from this list, I have a new perfectly functioning IMAP server.
The issue is that macOS Note.app does not sync with this server. However, iOS Notes.app *does* sync and work well with this IMAP server, and the differences between macOS and iOS Notes apps are well known. (Thanks, Apple.) https://apple.stackexchange.com/questions/265579/notes-wont-sync-via-imap-in... https://apple.stackexchange.com/questions/265579/notes-wont-sync-via-imap-in... https://jpmens.net/2015/09/28/experiments-using-imap-for-notes/ https://jpmens.net/2015/09/28/experiments-using-imap-for-notes/ …
I am migrating from a working Server.app system in which Notes.app syncing does work across all devices, and therefore have a working dovecot configuration from Server.app, although it does not contain an explicit configuration for the “Notes” mailbox.
I would greatly appreciate any help or pointers for these questions:
Does anyone have a working template for dovecot that works with macOS Notes.app? I’ve posted doveconf -n of both the new server and the old Server.app server below.
Would you all be able to provide advice on TO-DO’s and NOT-TO-DO’s for mailbox/name-space reconfiguration as I try to muck around with a nicely working, deployed IMAP server? I believe that the solution has something to do with name spaces and separators, and I’ve already changed the separator from the original ‘/’ to ‘.’ on the working server without success or obvious changes.
All the mail boxes on the new IMAP server look like this, *without* a prefix ‘.’:
ls -1 /var/mail/tld.domain.mail/username/mdbox/mailboxes/ Archive/ Deleted Messages/ Drafts/ INBOX/ Junk/ Notes/ Sent/ Trash/
All the mailboxes on the working Server.app mailserver look like this, *with* a prefix ‘.’:
$ sudo ls -1 /Library/Server/Mail/Data/mail/users/username .Archive .Deleted Messages .Drafts .Junk .Notes .Sent Messages cur … new subscriptions tmp
Here are the doveconf -n of the new server that doesn’t sync macOS Notes.app, and the old Server.app IMAP server that does:
New server dovecot -n [Note: original separator was set to ‘/’, now changed to ‘.’ in config.d/10-mail.conf]:
# 2.3.6 (): /opt/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.5.6 () # OS: Darwin 18.6.0 x86_64 apfs # Hostname: newhost.domain.tld auth_cache_size = 10 M auth_gssapi_hostname = $ALL auth_krb5_keytab = /opt/local/etc/dovecot/imap.keytab auth_mechanisms = plain gssapi auth_realms = newhost.domain.tld auth_socket_path = /opt/local/var/run/dovecot/auth-userdb auth_username_format = %Ln debug_log_path = /opt/local/var/log/mail/mail-debug.log disable_plaintext_auth = no first_valid_gid = 6 first_valid_uid = 6 imap_id_log = * imap_id_send = "name" * "version" * imap_idle_notify_interval = 29 mins imap_urlauth_submit_user = submit info_log_path = /opt/local/var/log/mail/mail-info.log last_valid_gid = 100 lda_mailbox_autocreate = yes log_path = /opt/local/var/log/mail/mail-err.log login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_access_groups = mail mail_attachment_dir = /private/var/mail/tld.domain.mail/attachments mail_attachment_fs = sis posix:mode=0666 mail_debug = yes mail_gid = mail mail_home = /private/var/mail/tld.domain.mail mail_location = mdbox:/private/var/mail/tld.domain.mail/%Ln/mdbox mail_log_prefix = "%s(pid %p user %u): " mail_plugins = quota zlib acl fts fts_solr fts_lucene mail_privileged_group = mail mail_uid = _dovecot managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapsieve vnd.dovecot.imapsieve mdbox_rotate_size = 200 M namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = create special_use = \Junk } mailbox Notspam_train { auto = create special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Spam_train { auto = create special_use = \Junk } mailbox Trash { auto = create special_use = \Trash } prefix = separator = . } passdb { driver = pam name = pam } plugin { fts = solr fts_autoindex = yes fts_autoindex_exclude = \Junk fts_solr = url=http://127.0.0.1:8983/solr/dovecot/ imapsieve_mailbox1_before = file:/opt/local/etc/dovecot/sieve/report-spam.sieve imapsieve_mailbox1_causes = COPY APPEND imapsieve_mailbox1_name = Spam_train imapsieve_mailbox2_before = file:/opt/local/etc/dovecot/sieve/report-ham.sieve imapsieve_mailbox2_causes = COPY APPEND imapsieve_mailbox2_name = Notspam_train mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid from subject size flags quota_grace = 10%% quota_rule = *:storage=16G quota_rule2 = Trash:storage=+256M quota_warning = storage=100%% quota-exceeded %u quota_warning2 = storage=80%% quota-warning %u recipient_delimiter = + sieve = /private/var/mail/tld.domain.mail/rules/%Ln/dovecot.sieve sieve_after = /opt/local/etc/dovecot/sieve-after.d sieve_before = /opt/local/etc/dovecot/sieve-before.d sieve_dir = /private/var/mail/tld.domain.mail/rules/%Ln/%u sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /opt/local/etc/dovecot/sieve sieve_plugins = sieve_imapsieve sieve_extprograms sieve_quota_max_storage = 50M } postmaster_address = postmaster@domain.tld protocols = imap lmtp sieve quota_full_tempfail = yes sendmail_path = /opt/local/sbin/sendmail service auth-worker { group = mail user = root } service auth { extra_groups = _keytabusers idle_kill = 15 mins unix_listener /opt/local/var/spool/postfix/private/auth { group = mail mode = 0660 user = _postfix } } service imap-login { inet_listener imap { address = 127.0.0.1, ::1 port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 6 service_count = 0 vsz_limit = 2 G } service imap { process_limit = 200 process_min_avail = 6 } service lmtp { unix_listener /opt/local/var/spool/postfix/private/dovecot-lmtp { group = mail mode = 0660 user = _postfix } } ssl = required ssl_ca =
Old Server.app server dovecot -n:
# 2.2.30.2 (c0c463e): /Library/Server/Mail/Config/dovecot/dovecot.conf # Pigeonhole version 0.4.18 (29cc74d) # OS: Darwin 17.7.0 x86_64 apfs aps_topic = com.apple.mail.XServer.d7d6581e-37ed-4a8a-8775-92c4c5a4bf28 auth_gssapi_hostname = $ALL auth_mechanisms = cram-md5 plain login gssapi auth_realms = host.domain.tld auth_socket_path = /var/run/dovecot/auth-userdb auth_username_format = %n debug_log_path = /Library/Logs/Mail/mail-debug.log default_internal_user = _dovecot default_login_user = _dovenull disable_plaintext_auth = no first_valid_gid = 6 first_valid_uid = 6 imap_id_log = * imap_id_send = "name" * "version" * imap_urlauth_submit_user = submit info_log_path = /Library/Logs/Mail/mail-info.log log_path = /Library/Logs/Mail/mail-err.log login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_access_groups = mail mail_attribute_dict = file:/Library/Server/Mail/Data/attributes/attributes.dict mail_location = maildir:/Library/Server/Mail/Data/mail/%u mail_log_prefix = "%s(pid %p user %u): " mail_plugins = quota zlib acl fts fts_sk managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext mdbox_rotate_size = 200 M namespace acl-mailboxes { list = children location = maildir:/Library/Server/Mail/Data/mail/users/%%u:INDEX=/Library/Server/Mail/Data/mail/shared/%%u prefix = shared.%%u. separator = . subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } namespace list-archives { list = children location = maildir:/Library/Server/Mail/Data/listserver/messages/archive/lists/%%u:INDEX=/Library/Server/Mail/Data/listserver/messages/archive/shared/%%u prefix = archives.%%u. separator = . subscriptions = no type = shared } passdb { driver = od } passdb { args = /Library/Server/Mail/Config/dovecot/submit.passdb driver = passwd-file } plugin { acl = vfile:/Library/Server/Mail/Config/dovecot/global-acls:cache_secs=300 acl_shared_dict = file:/Library/Server/Mail/Data/shared/shared-mailboxes fts = sk quota = maildir:User quota quota_warning = storage=100%% quota-exceeded %u quota_warning2 = storage=80%% quota-warning %u sieve = /Library/Server/Mail/Data/rules/%u/dovecot.sieve sieve_dir = /Library/Server/Mail/Data/rules/%u stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = postmaster@host.domain.tld protocols = imap pop3 lmtp sieve quota_full_tempfail = yes service auth { extra_groups = _keytabusers idle_kill = 15 mins unix_listener auth-userdb { user = _dovecot } } service dns_client { unix_listener dns-client { mode = 0600 } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } service_count = 0 } service imap { client_limit = 5 process_limit = 200 service_count = 0 } service indexer-worker { user = _dovecot } service lmtp { unix_listener lmtp { mode = 0600 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service pop3 { client_limit = 5 process_limit = 200 service_count = 0 } service quota-exceeded { executable = script /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/quota-exceeded.sh unix_listener quota-exceeded { group = mail mode = 0660 user = _dovecot } user = _dovecot } service quota-warning { executable = script /Applications/Server.app/Contents/ServerRoot/usr/libexec/dovecot/quota-warning.sh unix_listener quota-warning { group = mail mode = 0660 user = _dovecot } user = _dovecot } service stats { fifo_listener stats-mail { mode = 0600 user = _dovecot } } ssl = required ssl_ca =
- Steven Smith via dovecot:
The issue is that macOS Note.app does not sync with this server.
I'm syncing macOS Mojave's Notes.app with Dovecot without problems, so it can be done. Have you made sure that you activated both mail and notes in macOS' Internet Accounts preferences for this particular IMAP account? If so, is that IMAP account listed in the navigation bar of Notes.app (mine shows iCloud and the IMAP account)?
-Ralph
- Ralph, Steven Smith via dovecot:
The issue is that macOS Note.app does not sync with this server.
I'm syncing macOS Mojave's Notes.app with Dovecot without problems, so it can be done. Have you made sure that you activated both mail and notes in macOS' Internet Accounts preferences for this particular IMAP account? If so, is that IMAP account listed in the navigation bar of Notes.app (mine shows iCloud and the IMAP account)?
Thanks, yes, I’ve done all that and more, including:
- Setting the Path Prefix to blank, INBOX, INBOX., and ‘.’
- sqlite3 ~/Library/Containers/com.apple.Notes/Data/Library/Notes/NotesV7.storedata 'select * from ZACCOUNT;'
- sqlite3 ~/Library/Containers/com.apple.Notes/Data/Library/Notes/NotesV7.storedata “update ZACCOUNT set ZUSERNAME='email@example.com', ZHOSTNAME='imap.hostname.com',ZSERVERPATHPREFIX='INBOX.' where Z_PK=5;”
- Variations on the above
Reports of problems syncing Notes.app to a IMAP accounts are widespread.
I’m pretty sure it arises from some subtle dovecot configuration setting.
Would you mind posting your
doveconf -nso that we can compare to the non-working and working examples above? Steve
It does not appear that macOS Notes.app is authenticating to dovecot. I haven’t broken out wireshark yet, but I see in the dovecot logs that a working iOS access looks like this:
mail-debug.log (mailbox access):
May 30 20:16:39 imap(pid 28792 user username): Debug: Mailbox Notes: Mailbox open ed because: SELECT
mail-info.log (solr indexing of a newly added Note):
May 30 18:44:31 indexer-worker(pid 26589 user username): Info: Indexed 1 messages in Notes (UIDs 167..167)
There are no such log entries when I launch macOS Notes.app and toggle account activation in System Preferences>Internet Accounts.
It doesn’t look like macOS Notes.app is even trying to talk to dovecot.
- Steven Smith via dovecot:
I’m pretty sure it arises from some subtle dovecot configuration setting.
Based on your statement that Dovecot does not log connection attempts by Notes.app, I am not sure this assumption is correct.
Would you mind posting your
doveconf -nso that we can compare to the non-working and working examples above?
See attachment (I have only obfuscated directories and deleted some settings related to authentication).
-Ralph
# 2.3.5.2 (38c8f1daf): /path/to/dovecot.conf # Pigeonhole version 0.5.5 (2483b085) # OS: Linux 5.1.5-gentoo x86_64 Gentoo Base System release 2.6 auth_mechanisms = plain login auth_username_format = %Ln lmtp_save_to_detail_mailbox = yes mail_location = /path/to/maildir mail_plugins = notify managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext vnd.dovecot.debug imapsieve vnd.dovecot.imapsieve namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { auto = create special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = create special_use = \Trash } prefix = separator = / } passdb { args = /path/to/dovecot-ldap.conf.ext driver = ldap } plugin { imapsieve_mailbox1_before = file:/path/to/report-spam.sieve imapsieve_mailbox1_causes = COPY imapsieve_mailbox1_name = Junk imapsieve_mailbox2_before = file:/path/to/report-ham.sieve imapsieve_mailbox2_causes = COPY imapsieve_mailbox2_from = Junk imapsieve_mailbox2_name = * sieve = /path/to/sieve/%u.sieve sieve_extensions = +vnd.dovecot.debug sieve_global = /path/to/sieve/global sieve_global_extensions = +vnd.dovecot.pipe +vnd.dovecot.environment sieve_pipe_bin_dir = /path/to/sievepipe sieve_plugins = sieve_imapsieve sieve_extprograms } protocols = imap lmtp service auth { unix_listener /path/to/dovecot-auth { group = postfix mode = 0660 user = postfix } } service dict { unix_listener dict { mode = 0666 } } service lmtp { unix_listener /path/to/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } userdb { args = /path/to/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = notify sieve namespace inbox { location = mailbox Trash { autoexpunge = 1 weeks } prefix = } } protocol lda { mail_plugins = notify sieve } protocol imap { mail_plugins = notify imap_sieve namespace inbox { location = mailbox Trash { autoexpunge = 1 weeks } prefix = } }
Thank you very much for posting this. Looking at the diffs across configurations, I tried all these options, without success:
- service dns_client block
- turn off global separator variable
- auth_mechanisms login plain gssapi cram-md5
My dovecot configuration is the common denominator of multiple Notes.app clients not working, so I’m pretty sure I’ve configured it in a way that macOS Notes.app doesn’t like.
The only other difference I see is the ldap passwd and userdb driver.
Looks like I’ll have to wait for some inspiration and free time to break out wireshark to dig further.
Thanks again,
Steve
participants (3)
-
Aki Tuomi
-
Ralph Seichter
-
Steven Smith