<div dir="ltr"><div>⌂63% [ler@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX</div><div>⌂67% [ler@thebighonker.lerctr.org:~] 1 $ mail -s "test fts1970" <a href="mailto:ler@lerctr.org">ler@lerctr.org</a></div><div>test fts1970</div><div><br></div><div>test fts1970</div><div>.</div><div>EOT</div><div>[ler@thebighonker.lerctr.org:~] $ mailq</div><div>[ler@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX</div><div>Subject: test fts1970</div><div>test fts1970</div><div>test fts1970</div><div><br></div><div><br></div><div><div>Apr 15 04:29:03 thebighonker exim[49528]: 1hFxvD-000Csq-P6 <= <a href="mailto:ler@lerctr.org">ler@lerctr.org</a> U=ler P=local S=388</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Connect from local</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(<a href="http://ler@lerctr.org/49364">ler@lerctr.org/49364</a>): save: box=INBOX, uid=175402, msgid=<<a href="mailto:E1hFxvD-000Csq-P6@thebighonker.lerctr.org">E1hFxvD-000Csq-P6@thebighonker.lerctr.org</a>>, size=640, vsize=660, from=Larry Rosenman <<a href="mailto:ler@lerctr.org">ler@lerctr.org</a>>, subject=test fts1970, flags=()</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(<a href="http://ler@lerctr.org/49364">ler@lerctr.org/49364</a>): sieve: msgid=<<a href="mailto:E1hFxvD-000Csq-P6@thebighonker.lerctr.org">E1hFxvD-000Csq-P6@thebighonker.lerctr.org</a>>: stored mail into mailbox 'INBOX' (subject=test fts1970 from=<a href="mailto:ler@lerctr.org">ler@lerctr.org</a> size=660)</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Disconnect from local: Client has quit the connection (state=READY)</div><div>Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 => ler <<a href="mailto:ler@lerctr.org">ler@lerctr.org</a>> R=localuser T=dovecot_lmtp S=404 C="250 2.0.0 <<a href="mailto:ler@lerctr.org">ler@lerctr.org</a>> 6ACWMN9OtFzUwAAAu+mOrA Saved" QT=0s DT=0s</div><div>Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 Completed QT=0s</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: indexer-worker(<a href="http://ler@lerctr.org/49366">ler@lerctr.org/49366</a>): Indexed 1 messages in INBOX (UIDs 175402..175402)</div></div><div><br></div><div><br></div><div><div>⌂81% [ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX body 'fts1970'</div><div>⌂83% [ler@thebighonker.lerctr.org:~] $</div></div><div><br></div><div><br></div><div><div>⌂65% [ler@thebighonker.lerctr.org:~] 75 $ doveadm search -u <a href="mailto:ler@lerctr.org">ler@lerctr.org</a> mailbox INBOX body 'fts1970'</div><div>a53a143be44bda5bd4830000bbe98eac 175402</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm index -q INBOX</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX body 'fts1970'</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm fts rescan</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm index -q INBOX</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX body 'fts1970'</div><div>a53a143be44bda5bd4830000bbe98eac 175402</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm search -u <a href="mailto:ler@lerctr.org">ler@lerctr.org</a> mailbox INBOX body 'fts1970'</div><div>a53a143be44bda5bd4830000bbe98eac 175402</div><div>[ler@thebighonker.lerctr.org:~] $</div></div><div><br></div><div>So, yes, your hypothesis is correct.</div><div><br></div><div>Question: How can I make it consistent? </div><div><br></div><div>I have a script that runs on the first of the month that does archiving, and I have similar issues in that namespace:</div><div><div>⌂67% [ler@thebighonker.lerctr.org:~] $ cat bin/archive-mail</div><div>#!/bin/sh</div><div>PATH=$PATH:/usr/local/bin</div><div>#Expects to be run after midnight on the first of the month</div><div># to archive all the previous months mail</div><div>#Date Run:</div><div>TODAY=`date "+%Y-%m-%d"`</div><div>#last month in YYYY/MM</div><div>YEAR_LAST_MONTH=`date -v-1d "+%Y/%m"`</div><div>#1st of last month as 01-Mon-YYYY</div><div>FIRST_LAST_MONTH=`date -v-1d "+01-%b-%Y"`</div><div>echo 'TODAY=' ${TODAY}</div><div>echo 'YEAR_LAST_MONTH=' ${YEAR_LAST_MONTH}</div><div>echo 'FIRST_LAST_MONTH=' ${FIRST_LAST_MONTH}</div><div># get a list of all the mailboxes with at least one real message</div><div>doveadm -f tab mailbox status vsize \* 2>/dev/null |</div><div> sed -e 1d | sort -k 1,1 |</div><div> awk 'BEGIN {FS="\t"} {if ($2 > 0) print $1}' |</div><div>while read i</div><div>do</div><div> echo `date` start ${i}</div><div> doveadm mailbox create "ARCHIVE/${YEAR_LAST_MONTH}/${i}"</div><div> doveadm -f tab mailbox status messages "${i}"</div><div> doveadm move "ARCHIVE/${YEAR_LAST_MONTH}/${i}" mailbox \</div><div> "${i}" BEFORE ${TODAY} SINCE ${FIRST_LAST_MONTH}</div><div> doveadm -f tab mailbox status messages "${i}"</div><div> echo `date` done ${i}</div><div>done</div><div>⌂64% [ler@thebighonker.lerctr.org:~] $</div></div><div><br></div><div><br></div><div>The Exim config can be provided as well if necessary.</div><div><br></div><div>ler & <a href="mailto:ler@lerctr.org">ler@lerctr.org</a> *ARE THE SAME MAILBOX*</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 15, 2019 at 4:05 AM John Fawcett via dovecot <<a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15/04/2019 10:59, Larry Rosenman via
dovecot wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
I'll run a full test when I'm back in front of areal computer
vs. My phone.(in a few hours)<br>
<br>
</div>
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
Get <a href="https://aka.ms/ghei36" target="_blank">Outlook
for Android</a></div>
<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7233150499972651452divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
dovecot <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot-bounces@dovecot.org" target="_blank"><dovecot-bounces@dovecot.org></a> on behalf of John
Fawcett via dovecot <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot@dovecot.org" target="_blank"><dovecot@dovecot.org></a><br>
<b>Sent:</b> Monday, April 15, 2019 3:57:08 AM<br>
<b>To:</b> Dovecot Mailing List<br>
<b>Subject:</b> Re: SOLR/Index?</font>
<div> </div>
</div>
<div>
<div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15/04/2019 10:31, Larry Rosenman
via dovecot wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
It always shows the autoindex. And yes built from sources.
I'm the FreeBSD port maintainer for mail/docecot. This has
been happening for several releases.<br>
<br>
</div>
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
Get <a href="https://aka.ms/ghei36" target="_blank">Outlook for Android</a></div>
<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7233150499972651452divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
dovecot
<a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot-bounces@dovecot.org" target="_blank"><dovecot-bounces@dovecot.org></a>
on behalf of John Fawcett via dovecot
<a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot@dovecot.org" target="_blank"><dovecot@dovecot.org></a><br>
<b>Sent:</b> Monday, April 15, 2019 2:06:55 AM<br>
<b>To:</b> <a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:dovecot@dovecot.org" target="_blank">
dovecot@dovecot.org</a><br>
<b>Subject:</b> Re: SOLR/Index?</font>
<div> </div>
</div>
<div>
<div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15/04/2019 08:09, Larry
Rosenman via dovecot wrote:<br>
</div>
<blockquote type="cite">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
Note the hits after the fts rescan/index.<br>
<br>
</div>
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
<div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
Get <a href="https://aka.ms/ghei36" target="_blank">Outlook for Android</a></div>
<br>
</div>
<hr style="display:inline-block;width:98%">
<div id="gmail-m_-7233150499972651452divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Aki Tuomi
<a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:aki.tuomi@open-xchange.com" target="_blank"><aki.tuomi@open-xchange.com></a><br>
<b>Sent:</b> Monday, April 15, 2019 12:55:07 AM<br>
<b>To:</b> Larry Rosenman; John Fawcett<br>
<b>Cc:</b> Dovecot Mailing List<br>
<b>Subject:</b> Re: SOLR/Index?</font>
<div> </div>
</div>
<div>
<p><br>
</p>
<div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15.4.2019 3.33, Larry
Rosenman via dovecot wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">⌂72% [<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
$ doveadm search mailbox
lists/freebsd/ports-commiters body 'sysutils'<br>
</div>
<div dir="ltr">[<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
$ doveadm fts rescan</div>
<div dir="ltr">[<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
$ doveadm index -q lists/freebsd/ports-commiters</div>
<div dir="ltr">⌂64% [<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
$ tail -f /var/log/maillog</div>
<div dir="ltr">Apr 14 19:30:27 thebighonker
dovecot[2507]: imap-login: Disconnected (auth
failed, 1 attempts in 2 secs): user=<gwc>,
method=PLAIN, rip=180.180.217.124,
lip=192.147.25.65, TLS: Connection closed,
session=<EAk5woaGV4S0tNl8></div>
<div dir="ltr">Apr 14 19:30:28 thebighonker
dovecot[2507]: imap-login: Login:
user=<ler>, method=PLAIN,
rip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
lip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
mpid=14813, TLS,
session=<lr1mwoaGwYMgAQRwHw8DrQC73P/+UNkA></div>
<div dir="ltr">Apr 14 19:30:30 thebighonker
dovecot[2507]: imap(ler/14813): Logged out
in=12412 out=66691 fhc=0 fhb=0 fbc=0 fbb=0 del=0
exp=0 trash=0</div>
<div dir="ltr">Apr 14 19:30:54 thebighonker
exim[14846]: no host name found for IP address
23.100.68.192</div>
<div dir="ltr">Apr 14 19:30:55 thebighonker
exim[14846]: H=(<a href="http://DaVinci-MWare.prophet21lab.com" target="_blank">DaVinci-MWare.prophet21lab.com</a>)
[23.100.68.192]:52130 I=[192.147.25.65]:25
sender verify defer for <<a href="mailto:info@duke.org" target="_blank">info@duke.org</a>>:
host lookup did not complete</div>
<div dir="ltr">Apr 14 19:30:55 thebighonker
exim[14846]: H=(<a href="http://DaVinci-MWare.prophet21lab.com" target="_blank">DaVinci-MWare.prophet21lab.com</a>)
[23.100.68.192]:52130 I=[192.147.25.65]:25
F=<<a href="mailto:info@duke.org" target="_blank">info@duke.org</a>>
temporarily rejected RCPT <<a href="mailto:jpotyh@why.net" target="_blank">jpotyh@why.net</a>>:
Could not complete sender verify</div>
<div dir="ltr">Apr 14 19:31:04 thebighonker
dovecot[2507]: imap-login: Login:
user=<ctr>, method=PLAIN,
rip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
lip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
mpid=14910, TLS,
session=<bVWMxIaGJtogAQRwHw8DrQC73P/+UNkA></div>
<div dir="ltr">Apr 14 19:31:04 thebighonker
dovecot[2507]: imap(ctr/14910): Logged out
in=169 out=1711 fhc=0 fhb=0 fbc=0 fbb=0 del=0
exp=0 trash=0</div>
<div dir="ltr">Apr 14 19:31:16 thebighonker
exim[14911]: no host name found for IP address
80.253.235.35</div>
<div dir="ltr">Apr 14 19:31:19 thebighonker
dovecot[2507]: indexer-worker(ler/14919):
Indexed 1578 messages in
lists/freebsd/ports-commiters (UIDs
21067..22644)</div>
<div dir="ltr">^C</div>
<div dir="ltr">[<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
130 $ doveadm search mailbox
lists/freebsd/ports-commiters body 'sysutils/'</div>
</div>
</div>
</blockquote>
<p><br>
</p>
<p>Just minor nit, but you are searching for 'sysutils'
first, then 'sysutils/'. FTS does not do substring
searches by default.<br>
</p>
<p>Aki<br>
</p>
<blockquote type="cite">
<div dir="ltr"><br>
<div>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">-- </div>
</div>
<div dir="ltr" class="gmail-m_-7233150499972651452gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">Larry Rosenman
<a href="http://www.lerctr.org/~ler" target="_blank">
http://www.lerctr.org/~ler</a><br>
<span>Phone: <span title="Call
with Google Voice">
<span title="Call with Google
Voice"><span title="Call
with Google Voice"><span id="gmail-m_-7233150499972651452gc-number-162" class="gmail-m_-7233150499972651452gc-cs-link" title="Call with Google
Voice">+1 214-642-9640</span></span></span></span>
(c) E-Mail: </span><a href="mailto:larryrtx@gmail.com" target="_blank">larryrtx@gmail.com</a><br>
US Mail: 5708 Sabbia Dr, Round
Rock, TX 78665-2106</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</blockquote>
<p>Larry</p>
<p>just to be sure: are you running a standard unmodified
2.3.5.1 version which you built from source code?</p>
<p>I can see that first you search for sysutils, then do a
rescan and reindex (which is shown in the log) and then
you are able to find sysutils/.</p>
<p>It is better when doing these tests to search for the
same string before and after, just to eliminate too many
different factors in the test.</p>
<p>Nevertheless I did not see your logging for what happens
when you receive a test message containing sysutils/.
Dovecot should be outputing info about autoindexing given
your setup. Does it do that or does it give some other
message? Can you show those logs?<br>
</p>
<p>John<br>
</p>
</div>
</blockquote>
<p>Larry</p>
<p>Did you notice any difference between the logging for auto
indexing and the logging for indexing that you triggered
manually? Would you mind posting the auto indexing logging for
a message to that same user (ler)?<br>
</p>
<p>best regards</p>
<p>John<br>
</p>
</div>
</blockquote>
<p>ok, the hypothesis I'm trying to confirm is that auto indexing is
taking place under the full username ler@somedomain whereas in the
command line tests you did your are using user ler (as presumably
is roundcube).</p>
<p>John<br>
</p>
</div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Larry Rosenman <a href="http://www.lerctr.org/~ler" target="_blank">http://www.lerctr.org/~ler</a><br><span>Phone: <span title="Call with Google Voice"><span title="Call with Google Voice"><span title="Call with Google Voice">+1 214-642-9640</span></span></span> (c) E-Mail: </span><a href="mailto:larryrtx@gmail.com" target="_blank">larryrtx@gmail.com</a><br>US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106</div></div></div></div></div></div></div></div>