[Dovecot] meaning of parameter imap_idle_notify_interval??
Hello,
is my understanding of the parameter "imap_idle_notify_interval" right, if i say, it's the time between two point where dovecot tells the client he is here, and sould have nothing directly todo with the notification of new mails arrived.
I played with the values around. And the courios is, in thunderbird the folloing.
set to 30 seconds.
Thunderbird pop up around 25 seconds after the message is deliverd to the inbox.
set to 60 seconds.
Thunderbird pup up around 50 seconds after the message is deliverd to the inbox.
I switch over from archiveopteryx, where no such delay was.
Regards, Ingo
On Wed, 2009-10-21 at 21:28 +0200, Ingo Thierack wrote:
Hello,
is my understanding of the parameter "imap_idle_notify_interval" right, if i say, it's the time between two point where dovecot tells the client he is here, and sould have nothing directly todo with the notification of new mails arrived.
Right.
I played with the values around. And the courios is, in thunderbird the folloing.
set to 30 seconds.
Thunderbird pop up around 25 seconds after the message is deliverd to the inbox.
set to 60 seconds.
Thunderbird pup up around 50 seconds after the message is deliverd to the inbox.
I don't see how that setting would affect either of these.
Did you build Dovecot with inotify/dnotify/kqueue support? (What does dovecot --build-options show?) If not, mailbox_idle_check_interval is what controls how often new mails are checked.
Here are the requestet information. It was compiled with inotify which is the default on my system.
First I thought, it was the purpose, that the mailbox dirs was on an NFS. Then copied to the local disc. But the same.
[root@mail ~]# /opt/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl Mail storages: cydir dbox maildir mbox raw shared SQL drivers: postgresql Passdb: checkpassword pam passwd passwd-file shadow sql Userdb: nss passwd passwd-file prefetch sql static
[root@mail ~]# /opt/dovecot/sbin/dovecot -n # 1.2.6: /opt/dovecot/etc/dovecot.conf # OS: Linux 2.6.18-128.7.1.el5 i686 CentOS release 5.3 (Final) ext3 base_dir: /var/run/dovecot/ listen: *:143 ssl_listen: *:993 ssl_cert_file: /opt/dovecot/etc/ssl/certs/mail.ghostdive.org.crt ssl_key_file: /opt/dovecot/etc/ssl/private/mail.ghostdive.org.key disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable: /opt/dovecot/libexec/dovecot/imap-login login_greeting: ready. login_processes_count: 10 mail_max_userip_connections: 50 mail_location: maildir:/mnt/mail/%u/Maildir:INDEX=/var/spool/indexes/%u mailbox_idle_check_interval: 2 dbox_rotate_size: 4096 dbox_rotate_days: 30 mail_executable: /opt/dovecot/libexec/dovecot/rawlog -b /opt/dovecot/libexec/dovecot/imap imap_idle_notify_interval: 30 lda: postmaster_address: postmaster@ghostdive.org hostname: mail.ghostdive.org mail_plugins: sieve mail_plugin_dir: /opt/dovecot/lib/dovecot/lda sendmail_path: /opt/postfix26/sbin/sendmail auth default: mechanisms: plain cram-md5 digest-md5 passdb: driver: sql args: /opt/dovecot/etc/dovecot-sql.conf userdb: driver: sql args: /opt/dovecot/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: mailaccess group: mailaccess
Timo Sirainen schrieb:
On Wed, 2009-10-21 at 21:28 +0200, Ingo Thierack wrote:
Hello,
is my understanding of the parameter "imap_idle_notify_interval" right, if i say, it's the time between two point where dovecot tells the client he is here, and sould have nothing directly todo with the notification of new mails arrived.
Right.
I played with the values around. And the courios is, in thunderbird the folloing.
set to 30 seconds.
Thunderbird pop up around 25 seconds after the message is deliverd to the inbox.
set to 60 seconds.
Thunderbird pup up around 50 seconds after the message is deliverd to the inbox.
I don't see how that setting would affect either of these.
Did you build Dovecot with inotify/dnotify/kqueue support? (What does dovecot --build-options show?) If not, mailbox_idle_check_interval is what controls how often new mails are checked.
Hello,
once more again.
upgrade the server to the latest kernel and libs (namely CentOS 5.4).
Same problem.
Set notify interval to 180 seconds. The message appears without intervention around 140 seconds after it was delivered to the inbox.
2009-10-26 21:01:09 | dovecot: | deliver(ingoiwt@ghostdive.org): sieve: msgid=ac2e3a8f0910261301r4d1f38dfx83e6d01d29394fd3@mail.gmail.com: stored mail into mailbox 'INBOX'
In thunderbird it appears around 21:03:30
It looks like, only when dovecot sends the message, thunderbird will refresh its view.
At the moment, i set it to 10 Seconds. But this will be an overkill for an mobilconnection :-P
[root@mail log]# /opt/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl Mail storages: cydir dbox maildir mbox raw shared SQL drivers: postgresql Passdb: checkpassword pam passwd passwd-file shadow sql Userdb: nss passwd passwd-file prefetch sql static
[root@mail log]# /opt/dovecot/sbin/dovecot -n # 1.2.6: /opt/dovecot/etc/dovecot.conf # OS: Linux 2.6.18-164.el5 i686 CentOS release 5.4 (Final) ext3 base_dir: /var/run/dovecot/ listen: *:143 ssl_listen: *:993 ssl_cert_file: /opt/dovecot/etc/ssl/certs/mail.ghostdive.org.crt ssl_key_file: /opt/dovecot/etc/ssl/private/mail.ghostdive.org.key disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable: /opt/dovecot/libexec/dovecot/imap-login login_greeting: ready. login_processes_count: 10 mail_max_userip_connections: 50 mail_location: maildir:/mnt/mail/%u/Maildir:INDEX=/var/spool/indexes/%u mailbox_idle_check_interval: 2 dbox_rotate_size: 4096 dbox_rotate_days: 30 imap_client_workarounds: tb-extra-mailbox-sep imap_idle_notify_interval: 180 lda: postmaster_address: postmaster@ghostdive.org hostname: mail.ghostdive.org mail_plugins: sieve mail_plugin_dir: /opt/dovecot/lib/dovecot/lda sendmail_path: /opt/postfix26/sbin/sendmail auth default: mechanisms: plain cram-md5 digest-md5 passdb: driver: sql args: /opt/dovecot/etc/dovecot-sql.conf userdb: driver: sql args: /opt/dovecot/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: mailaccess group: mailaccess
Ingo Thierack wrote:
Here are the requestet information. It was compiled with inotify which is the default on my system.
First I thought, it was the purpose, that the mailbox dirs was on an NFS. Then copied to the local disc. But the same.
[root@mail ~]# /opt/dovecot/sbin/dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl Mail storages: cydir dbox maildir mbox raw shared SQL drivers: postgresql Passdb: checkpassword pam passwd passwd-file shadow sql Userdb: nss passwd passwd-file prefetch sql static
[root@mail ~]# /opt/dovecot/sbin/dovecot -n # 1.2.6: /opt/dovecot/etc/dovecot.conf # OS: Linux 2.6.18-128.7.1.el5 i686 CentOS release 5.3 (Final) ext3 base_dir: /var/run/dovecot/ listen: *:143 ssl_listen: *:993 ssl_cert_file: /opt/dovecot/etc/ssl/certs/mail.ghostdive.org.crt ssl_key_file: /opt/dovecot/etc/ssl/private/mail.ghostdive.org.key disable_plaintext_auth: no verbose_ssl: yes login_dir: /var/run/dovecot/login login_executable: /opt/dovecot/libexec/dovecot/imap-login login_greeting: ready. login_processes_count: 10 mail_max_userip_connections: 50 mail_location: maildir:/mnt/mail/%u/Maildir:INDEX=/var/spool/indexes/%u mailbox_idle_check_interval: 2 dbox_rotate_size: 4096 dbox_rotate_days: 30 mail_executable: /opt/dovecot/libexec/dovecot/rawlog -b /opt/dovecot/libexec/dovecot/imap imap_idle_notify_interval: 30 lda: postmaster_address: postmaster@ghostdive.org hostname: mail.ghostdive.org mail_plugins: sieve mail_plugin_dir: /opt/dovecot/lib/dovecot/lda sendmail_path: /opt/postfix26/sbin/sendmail auth default: mechanisms: plain cram-md5 digest-md5 passdb: driver: sql args: /opt/dovecot/etc/dovecot-sql.conf userdb: driver: sql args: /opt/dovecot/etc/dovecot-sql.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 384 user: mailaccess group: mailaccess
Timo Sirainen schrieb:
On Wed, 2009-10-21 at 21:28 +0200, Ingo Thierack wrote:
Hello,
is my understanding of the parameter "imap_idle_notify_interval" right, if i say, it's the time between two point where dovecot tells the client he is here, and sould have nothing directly todo with the notification of new mails arrived.
Right.
I played with the values around. And the courios is, in thunderbird the folloing.
set to 30 seconds.
Thunderbird pop up around 25 seconds after the message is deliverd to the inbox.
set to 60 seconds.
Thunderbird pup up around 50 seconds after the message is deliverd to the inbox.
I don't see how that setting would affect either of these.
Did you build Dovecot with inotify/dnotify/kqueue support? (What does dovecot --build-options show?) If not, mailbox_idle_check_interval is what controls how often new mails are checked.
On Mon, 2009-10-26 at 21:20 +0100, Ingo Thierack wrote:
Set notify interval to 180 seconds. The message appears without intervention around 140 seconds after it was delivered to the inbox.
See if this helps: http://hg.dovecot.org/dovecot-1.2/rev/abbb153d4643
sorry to say, no change. Takes more then 2 minutes until tb register there is a new mail.
I'm the only with this behavior? As sayd before, had no issues with archiveopteryx-imap on this. So I think, there is no problem with tb.
Could there be an problem with the inotify implementation in the CentOS-Kernel?
Timo Sirainen schrieb:
On Mon, 2009-10-26 at 21:20 +0100, Ingo Thierack wrote:
Set notify interval to 180 seconds. The message appears without intervention around 140 seconds after it was delivered to the inbox.
See if this helps: http://hg.dovecot.org/dovecot-1.2/rev/abbb153d4643
On Tue, 2009-10-27 at 09:55 +0100, Ingo Thierack wrote:
sorry to say, no change. Takes more then 2 minutes until tb register there is a new mail.
I'm the only with this behavior?
At least only one who has noticed it..
Could there be an problem with the inotify implementation in the CentOS-Kernel?
I doubt it, but you could try if the attached patch logs a line whenever a new message arrives. If it does get logged and Dovecot doesn't send an EXISTS line to Thunderbird immediately, there's a bug. If that does happen, the next step could be to try it manually instead of using Thunderbird, e.g.:
telnet localhost 143 a login username password b select inbox c idle
and then see if the EXISTS gets sent.
tried the telnet approach. patched in the logline.
[ingo@mail ~]$ telnet localhost 143 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'.
- OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN AUTH=CRAM-MD5 AUTH=DIGEST-MD5] ready. a login xxxx xxxx a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted.
- 13 EXISTS
- 0 RECENT
- OK [UIDVALIDITY 1252958392] UIDs valid
- OK [UIDNEXT 37] Predicted next UID
- OK [HIGHESTMODSEQ 81] Highest b OK [READ-WRITE] Select completed. c idle
- idling
after idling displayed, i sent a message to the account.
see the message delivered to the inbox in my syslog
2009-10-27 23:35:20 | dovecot: | deliver(ingo@ghostdive.org): sieve: msgid=ac2e3a8f0910271535q4c5096a8w56d31e117cb7bf76@mail.gmail.com: stored mail into mailbox 'INBOX'
in logfile:
<<
You seem to have kind of mixed everything into one so I don't know what happened. Rawlog (*.in, *.out files) isn't the same as where my patch was logging the message, it went to Dovecot's error log http://wiki.dovecot.org/Logging. Also do you mean there was no EXISTS reply sent at all after IDLE was started in either rawlog or telnet session?
So the most important thing: After deliver logs the message, how long does it take for "inotify triggered" message to get written to Dovecot's log file? Or does it never get logged there? (Make sure you're looking at the log file where dovecot --log-error writes "This is Dovecot's warning log.")
On Tue, 2009-10-27 at 23:42 +0100, Ingo Thierack wrote:
tried the telnet approach. patched in the logline.
[ingo@mail ~]$ telnet localhost 143 Trying 127.0.0.1... Connected to localhost.localdomain (127.0.0.1). Escape character is '^]'.
- OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN AUTH=CRAM-MD5 AUTH=DIGEST-MD5] ready. a login xxxx xxxx a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted.
- 13 EXISTS
- 0 RECENT
- OK [UIDVALIDITY 1252958392] UIDs valid
- OK [UIDNEXT 37] Predicted next UID
- OK [HIGHESTMODSEQ 81] Highest b OK [READ-WRITE] Select completed. c idle
- idling
after idling displayed, i sent a message to the account.
see the message delivered to the inbox in my syslog
2009-10-27 23:35:20 | dovecot: | deliver(ingo@ghostdive.org): sieve: msgid=ac2e3a8f0910271535q4c5096a8w56d31e117cb7bf76@mail.gmail.com: stored mail into mailbox 'INBOX'
in logfile: <<
a little bit late but now.
As far as I can see, no EXISTS, after idling
Even can't see the additional output in the logmessages.
Looks like the code is never reached?
telnet:
- OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE STARTTLS AUTH=PLAIN AUTH=CRAM-MD5 AUTH=DIGEST-MD5] ready. a login xxx xxxxx a OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE SORT THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT IDLE CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH] Logged in b select inbox
- FLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk)
- OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk \*)] Flags permitted.
- 6 EXISTS
- 0 RECENT
- OK [UIDVALIDITY 1252958392] UIDs valid
- OK [UIDNEXT 41] Predicted next UID
- OK [HIGHESTMODSEQ 104] Highest b OK [READ-WRITE] Select completed. c idle
- idling
- OK Still here
- OK Still here
logfile
Nov 04 12:20:47 dovecot: Info: This is Dovecot's info log Nov 04 12:20:47 dovecot: Warning: This is Dovecot's warning log Nov 04 12:20:47 dovecot: Error: This is Dovecot's error log Nov 04 12:20:47 dovecot: Fatal: This is Dovecot's fatal log Nov 04 12:25:01 IMAP(ingo): Info: Effective uid=20000, gid=20000, home=/mnt/mail/home/ingo Nov 04 12:25:01 IMAP(ingo): Info: dbox: data=/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo Nov 04 12:25:01 IMAP(ingo): Info: fs: root=/mnt/mail/mail/ingo, index=/var/spool/indexes/ingo, control=, inbox= Nov 04 12:25:01 imap-login: Info: Login: user=<ingo>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, secured Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: Loading modules from directory: /opt/dovecot/lib/dovecot/lda Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: Module loaded: /opt/dovecot/lib/dovecot/lda/lib90_sieve_plugin.so Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: home=/mnt/mail/home/ingo Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: mail=dbox:/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: uid=20000 Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: gid=20000 Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: dbox: data=/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: fs: root=/mnt/mail/mail/ingo, index=/var/spool/indexes/ingo, control=, inbox= Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: sieve: using sieve path for user's script: /mnt/mail/home/ingo/.dovecot.sieve Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: sieve: opening script /mnt/mail/home/ingo/.dovecot.sieve Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: sieve: executing compiled script /mnt/mail/home/ingo/.dovecot.sieve Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: sieve: msgid=OF7BC0B05A.AEFFD6E5-ONC1257664.003EC4A0-C1257664.003ECD23@conrad.de: forwarded to ingo@a263722.exchange-mail.eu Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: sieve: msgid=OF7BC0B05A.AEFFD6E5-ONC1257664.003EC4A0-C1257664.003ECD23@conrad.de: stored mail into mailbox 'INBOX'
after that nothing more. Must I change some switch?
raw-log in
<<
<< raw-log out <<
<<<* FLAGS (\Answered \Flagged \Deleted \Seen \Draft
NonJunk) OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft NonJunk
\*)] Flags permitted. 6
EXISTS 0
RECENT OK [UIDVALIDITY 1252958392] UIDs
valid OK [UIDNEXT 41] Predicted next
UID OK [HIGHESTMODSEQ 104]
Highest b OK [READ-WRITE] Select
completed. <<<+
idling <<<* OK Still
here <<<* OK Still
here <<<* OK Still
here Timo Sirainen schrieb: You seem to have kind of mixed everything into one so I don't know what
happened. Rawlog (*.in, *.out files) isn't the same as where my patch
was logging the message, it went to Dovecot's error log
http://wiki.dovecot.org/Logging. Also do you mean there was no EXISTS
reply sent at all after IDLE was started in either rawlog or telnet
session? So the most important thing: After deliver logs the message, how long
does it take for "inotify triggered" message to get written to Dovecot's
log file? Or does it never get logged there? (Make sure you're looking
at the log file where dovecot --log-error writes "This is Dovecot's
warning log.")
On Wed, 2009-11-04 at 12:48 +0100, Ingo Thierack wrote:
mail_location: maildir:/mnt/mail/%u/Maildir:INDEX=/var/spool/indexes/%u .. Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: mail=dbox:/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo
You're using dbox instead of maildir! That's the reason. I haven't looked all that closely yet how nicely dbox works with IDLE.
And I switched extra to dbox...
Ok, any chance you take in the near future a look at it, or should I switch to maildir? Can i convert dbox to maildir?
Timo Sirainen wrote:
On Wed, 2009-11-04 at 12:48 +0100, Ingo Thierack wrote:
mail_location: maildir:/mnt/mail/%u/Maildir:INDEX=/var/spool/indexes/%u
...
Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: mail=dbox:/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo
You're using dbox instead of maildir! That's the reason. I haven't looked all that closely yet how nicely dbox works with IDLE.
On Thu, 2009-11-05 at 20:00 +0100, Ingo Thierack wrote:
Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: mail=dbox:/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo
Looks like the problem was that you had index files in a different directory than mail files. This should fix it: http://hg.dovecot.org/dovecot-1.2/rev/68af994e7f3c
Wow, fast work.
Tested it, and it looks like, it works now as expected. Set the interval to 20 minutes. Thunderbird popup immediately as a new messages arrive.
Yes, i have the index files on another drive/directory. And now, i'll put the mails back to an nfs-share, and let the index files on the local storage, as i configured originally. The purpose was, the nfs is only attached with 100mbit, but large, and i can backup it fast. So the indexfiles local for faster access, and the mails on the remote.
Thanks for you fast help.
Timo Sirainen schrieb:
On Thu, 2009-11-05 at 20:00 +0100, Ingo Thierack wrote:
Nov 04 12:26:28 deliver(ingo@ghostdive.org): Info: auth input: mail=dbox:/mnt/mail/mail/ingo:INDEX=/var/spool/indexes/ingo
Looks like the problem was that you had index files in a different directory than mail files. This should fix it: http://hg.dovecot.org/dovecot-1.2/rev/68af994e7f3c
participants (2)
-
Ingo Thierack
-
Timo Sirainen