<div dir="ltr"><div>⌂63% [ler@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX</div><div>⌂67% [ler@thebighonker.lerctr.org:~] 1 $ mail -s "test fts1970" <a href="mailto:ler@lerctr.org">ler@lerctr.org</a></div><div>test fts1970</div><div><br></div><div>test fts1970</div><div>.</div><div>EOT</div><div>[ler@thebighonker.lerctr.org:~] $ mailq</div><div>[ler@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX</div><div>Subject: test fts1970</div><div>test fts1970</div><div>test fts1970</div><div><br></div><div><br></div><div><div>Apr 15 04:29:03 thebighonker exim[49528]: 1hFxvD-000Csq-P6 <= <a href="mailto:ler@lerctr.org">ler@lerctr.org</a> U=ler P=local S=388</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Connect from local</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(<a href="http://ler@lerctr.org/49364">ler@lerctr.org/49364</a>): save: box=INBOX, uid=175402, msgid=<<a href="mailto:E1hFxvD-000Csq-P6@thebighonker.lerctr.org">E1hFxvD-000Csq-P6@thebighonker.lerctr.org</a>>, size=640, vsize=660, from=Larry Rosenman <<a href="mailto:ler@lerctr.org">ler@lerctr.org</a>>, subject=test fts1970, flags=()</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(<a href="http://ler@lerctr.org/49364">ler@lerctr.org/49364</a>): sieve: msgid=<<a href="mailto:E1hFxvD-000Csq-P6@thebighonker.lerctr.org">E1hFxvD-000Csq-P6@thebighonker.lerctr.org</a>>: stored mail into mailbox 'INBOX' (subject=test fts1970 from=<a href="mailto:ler@lerctr.org">ler@lerctr.org</a> size=660)</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Disconnect from local: Client has quit the connection (state=READY)</div><div>Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 => ler <<a href="mailto:ler@lerctr.org">ler@lerctr.org</a>> R=localuser T=dovecot_lmtp S=404 C="250 2.0.0 <<a href="mailto:ler@lerctr.org">ler@lerctr.org</a>> 6ACWMN9OtFzUwAAAu+mOrA Saved" QT=0s DT=0s</div><div>Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 Completed QT=0s</div><div>Apr 15 04:29:03 thebighonker dovecot[2507]: indexer-worker(<a href="http://ler@lerctr.org/49366">ler@lerctr.org/49366</a>): Indexed 1 messages in INBOX (UIDs 175402..175402)</div></div><div><br></div><div><br></div><div><div>⌂81% [ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX  body 'fts1970'</div><div>⌂83% [ler@thebighonker.lerctr.org:~] $</div></div><div><br></div><div><br></div><div><div>⌂65% [ler@thebighonker.lerctr.org:~] 75 $ doveadm search -u <a href="mailto:ler@lerctr.org">ler@lerctr.org</a>  mailbox INBOX body 'fts1970'</div><div>a53a143be44bda5bd4830000bbe98eac 175402</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm index -q INBOX</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX  body 'fts1970'</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm fts rescan</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm index -q INBOX</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX  body 'fts1970'</div><div>a53a143be44bda5bd4830000bbe98eac 175402</div><div>[ler@thebighonker.lerctr.org:~] $ doveadm search -u <a href="mailto:ler@lerctr.org">ler@lerctr.org</a>  mailbox INBOX body 'fts1970'</div><div>a53a143be44bda5bd4830000bbe98eac 175402</div><div>[ler@thebighonker.lerctr.org:~] $</div></div><div><br></div><div>So, yes, your hypothesis is correct.</div><div><br></div><div>Question: How can I make it consistent?  </div><div><br></div><div>I have a script that runs on the first of the month that does archiving, and I have similar issues in that namespace:</div><div><div>⌂67% [ler@thebighonker.lerctr.org:~] $ cat bin/archive-mail</div><div>#!/bin/sh</div><div>PATH=$PATH:/usr/local/bin</div><div>#Expects to be run after midnight on the first of the month</div><div>#  to archive all the previous months mail</div><div>#Date Run:</div><div>TODAY=`date "+%Y-%m-%d"`</div><div>#last month in YYYY/MM</div><div>YEAR_LAST_MONTH=`date -v-1d "+%Y/%m"`</div><div>#1st of last month as 01-Mon-YYYY</div><div>FIRST_LAST_MONTH=`date -v-1d "+01-%b-%Y"`</div><div>echo 'TODAY=' ${TODAY}</div><div>echo 'YEAR_LAST_MONTH=' ${YEAR_LAST_MONTH}</div><div>echo 'FIRST_LAST_MONTH=' ${FIRST_LAST_MONTH}</div><div># get a list of all the mailboxes with at least one real message</div><div>doveadm -f tab mailbox status vsize \* 2>/dev/null |</div><div>        sed -e 1d | sort -k 1,1 |</div><div>        awk  'BEGIN {FS="\t"} {if ($2 > 0)  print $1}' |</div><div>while read i</div><div>do</div><div>   echo `date` start ${i}</div><div>   doveadm mailbox create "ARCHIVE/${YEAR_LAST_MONTH}/${i}"</div><div>   doveadm -f tab mailbox status messages "${i}"</div><div>   doveadm move "ARCHIVE/${YEAR_LAST_MONTH}/${i}" mailbox \</div><div>            "${i}" BEFORE ${TODAY} SINCE ${FIRST_LAST_MONTH}</div><div>   doveadm -f tab mailbox status messages "${i}"</div><div>   echo `date` done  ${i}</div><div>done</div><div>⌂64% [ler@thebighonker.lerctr.org:~] $</div></div><div><br></div><div><br></div><div>The Exim config can be provided as well if necessary.</div><div><br></div><div>ler & <a href="mailto:ler@lerctr.org">ler@lerctr.org</a> *ARE THE SAME MAILBOX*</div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Apr 15, 2019 at 4:05 AM John Fawcett via dovecot <<a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
  
    
  
  <div bgcolor="#FFFFFF">
    <div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15/04/2019 10:59, Larry Rosenman via
      dovecot wrote:<br>
    </div>
    <blockquote type="cite">
      
      
      <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
        I'll run a full test when I'm back in front of areal computer
        vs. My phone.(in a few hours)<br>
        <br>
      </div>
      <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
        <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
          Get <a href="https://aka.ms/ghei36" target="_blank">Outlook
            for Android</a></div>
        <br>
      </div>
      <hr style="display:inline-block;width:98%">
      <div id="gmail-m_-7233150499972651452divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
          dovecot <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot-bounces@dovecot.org" target="_blank"><dovecot-bounces@dovecot.org></a> on behalf of John
          Fawcett via dovecot <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot@dovecot.org" target="_blank"><dovecot@dovecot.org></a><br>
          <b>Sent:</b> Monday, April 15, 2019 3:57:08 AM<br>
          <b>To:</b> Dovecot Mailing List<br>
          <b>Subject:</b> Re: SOLR/Index?</font>
        <div> </div>
      </div>
      <div>
        <div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15/04/2019 10:31, Larry Rosenman
          via dovecot wrote:<br>
        </div>
        <blockquote type="cite">
          
          <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
            It always shows the autoindex. And yes built from sources. 
            I'm the FreeBSD port maintainer for mail/docecot.  This has
            been happening for several releases.<br>
            <br>
          </div>
          <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
            <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
              Get <a href="https://aka.ms/ghei36" target="_blank">Outlook for Android</a></div>
            <br>
          </div>
          <hr style="display:inline-block;width:98%">
          <div id="gmail-m_-7233150499972651452divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b>
              dovecot
              <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot-bounces@dovecot.org" target="_blank"><dovecot-bounces@dovecot.org></a>
              on behalf of John Fawcett via dovecot
              <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:dovecot@dovecot.org" target="_blank"><dovecot@dovecot.org></a><br>
              <b>Sent:</b> Monday, April 15, 2019 2:06:55 AM<br>
              <b>To:</b> <a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:dovecot@dovecot.org" target="_blank">
                dovecot@dovecot.org</a><br>
              <b>Subject:</b> Re: SOLR/Index?</font>
            <div> </div>
          </div>
          <div>
            <div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15/04/2019 08:09, Larry
              Rosenman via dovecot wrote:<br>
            </div>
            <blockquote type="cite">
              
              <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
                Note the hits after the fts rescan/index.<br>
                <br>
              </div>
              <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
                <div dir="auto" style="direction:ltr;margin:0px;padding:0px;font-family:sans-serif;font-size:11pt;color:black">
                  Get <a href="https://aka.ms/ghei36" target="_blank">Outlook for Android</a></div>
                <br>
              </div>
              <hr style="display:inline-block;width:98%">
              <div id="gmail-m_-7233150499972651452divRplyFwdMsg" dir="ltr"><font style="font-size:11pt" face="Calibri, sans-serif" color="#000000"><b>From:</b> Aki Tuomi
                  <a class="gmail-m_-7233150499972651452moz-txt-link-rfc2396E" href="mailto:aki.tuomi@open-xchange.com" target="_blank"><aki.tuomi@open-xchange.com></a><br>
                  <b>Sent:</b> Monday, April 15, 2019 12:55:07 AM<br>
                  <b>To:</b> Larry Rosenman; John Fawcett<br>
                  <b>Cc:</b> Dovecot Mailing List<br>
                  <b>Subject:</b> Re: SOLR/Index?</font>
                <div> </div>
              </div>
              <div>
                <p><br>
                </p>
                <div class="gmail-m_-7233150499972651452moz-cite-prefix">On 15.4.2019 3.33, Larry
                  Rosenman via dovecot wrote:<br>
                </div>
                <blockquote type="cite">
                  <div dir="ltr">
                    <div dir="ltr">
                      <div dir="ltr">⌂72% [<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
                        $ doveadm search mailbox
                        lists/freebsd/ports-commiters  body 'sysutils'<br>
                      </div>
                      <div dir="ltr">[<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
                        $ doveadm fts rescan</div>
                      <div dir="ltr">[<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
                        $ doveadm index -q lists/freebsd/ports-commiters</div>
                      <div dir="ltr">⌂64% [<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
                        $ tail -f /var/log/maillog</div>
                      <div dir="ltr">Apr 14 19:30:27 thebighonker
                        dovecot[2507]: imap-login: Disconnected (auth
                        failed, 1 attempts in 2 secs): user=<gwc>,
                        method=PLAIN, rip=180.180.217.124,
                        lip=192.147.25.65, TLS: Connection closed,
                        session=<EAk5woaGV4S0tNl8></div>
                      <div dir="ltr">Apr 14 19:30:28 thebighonker
                        dovecot[2507]: imap-login: Login:
                        user=<ler>, method=PLAIN,
                        rip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
                        lip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
                        mpid=14813, TLS,
                        session=<lr1mwoaGwYMgAQRwHw8DrQC73P/+UNkA></div>
                      <div dir="ltr">Apr 14 19:30:30 thebighonker
                        dovecot[2507]: imap(ler/14813): Logged out
                        in=12412 out=66691 fhc=0 fhb=0 fbc=0 fbb=0 del=0
                        exp=0 trash=0</div>
                      <div dir="ltr">Apr 14 19:30:54 thebighonker
                        exim[14846]: no host name found for IP address
                        23.100.68.192</div>
                      <div dir="ltr">Apr 14 19:30:55 thebighonker
                        exim[14846]: H=(<a href="http://DaVinci-MWare.prophet21lab.com" target="_blank">DaVinci-MWare.prophet21lab.com</a>)
                        [23.100.68.192]:52130 I=[192.147.25.65]:25
                        sender verify defer for <<a href="mailto:info@duke.org" target="_blank">info@duke.org</a>>:
                        host lookup did not complete</div>
                      <div dir="ltr">Apr 14 19:30:55 thebighonker
                        exim[14846]: H=(<a href="http://DaVinci-MWare.prophet21lab.com" target="_blank">DaVinci-MWare.prophet21lab.com</a>)
                        [23.100.68.192]:52130 I=[192.147.25.65]:25
                        F=<<a href="mailto:info@duke.org" target="_blank">info@duke.org</a>>
                        temporarily rejected RCPT <<a href="mailto:jpotyh@why.net" target="_blank">jpotyh@why.net</a>>:
                        Could not complete sender verify</div>
                      <div dir="ltr">Apr 14 19:31:04 thebighonker
                        dovecot[2507]: imap-login: Login:
                        user=<ctr>, method=PLAIN,
                        rip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
                        lip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
                        mpid=14910, TLS,
                        session=<bVWMxIaGJtogAQRwHw8DrQC73P/+UNkA></div>
                      <div dir="ltr">Apr 14 19:31:04 thebighonker
                        dovecot[2507]: imap(ctr/14910): Logged out
                        in=169 out=1711 fhc=0 fhb=0 fbc=0 fbb=0 del=0
                        exp=0 trash=0</div>
                      <div dir="ltr">Apr 14 19:31:16 thebighonker
                        exim[14911]: no host name found for IP address
                        80.253.235.35</div>
                      <div dir="ltr">Apr 14 19:31:19 thebighonker
                        dovecot[2507]: indexer-worker(ler/14919):
                        Indexed 1578 messages in
                        lists/freebsd/ports-commiters (UIDs
                        21067..22644)</div>
                      <div dir="ltr">^C</div>
                      <div dir="ltr">[<a class="gmail-m_-7233150499972651452moz-txt-link-abbreviated" href="mailto:ler@thebighonker.lerctr.org:~" target="_blank">ler@thebighonker.lerctr.org:~</a>]
                        130 $ doveadm search mailbox
                        lists/freebsd/ports-commiters  body 'sysutils/'</div>
                    </div>
                  </div>
                </blockquote>
                <p><br>
                </p>
                <p>Just minor nit, but you are searching for 'sysutils'
                  first, then 'sysutils/'. FTS does not do substring
                  searches by default.<br>
                </p>
                <p>Aki<br>
                </p>
                <blockquote type="cite">
                  <div dir="ltr"><br>
                    <div>
                      <div class="gmail_quote">
                        <div dir="ltr" class="gmail_attr">-- </div>
                      </div>
                      <div dir="ltr" class="gmail-m_-7233150499972651452gmail_signature">
                        <div dir="ltr">
                          <div>
                            <div dir="ltr">
                              <div>
                                <div dir="ltr">
                                  <div>
                                    <div dir="ltr">Larry Rosenman       
                                                   <a href="http://www.lerctr.org/~ler" target="_blank">
                                        http://www.lerctr.org/~ler</a><br>
                                      <span>Phone: <span title="Call
                                          with Google Voice">
                                          <span title="Call with Google
                                            Voice"><span title="Call
                                              with Google Voice"><span id="gmail-m_-7233150499972651452gc-number-162" class="gmail-m_-7233150499972651452gc-cs-link" title="Call with Google
                                                Voice">+1 214-642-9640</span></span></span></span>
                                        (c)     E-Mail: </span><a href="mailto:larryrtx@gmail.com" target="_blank">larryrtx@gmail.com</a><br>
                                      US Mail: 5708 Sabbia Dr, Round
                                      Rock, TX 78665-2106</div>
                                  </div>
                                </div>
                              </div>
                            </div>
                          </div>
                        </div>
                      </div>
                    </div>
                  </div>
                </blockquote>
              </div>
            </blockquote>
            <p>Larry</p>
            <p>just to be sure: are you running a standard unmodified
              2.3.5.1 version which you built from source code?</p>
            <p>I can see that first you search for sysutils, then do a
              rescan and reindex (which is shown in the log) and then
              you are able to find sysutils/.</p>
            <p>It is better when doing these tests to search for the
              same string before and after, just to eliminate too many
              different factors in the test.</p>
            <p>Nevertheless I did not see your logging for what happens
              when you receive a test message containing sysutils/.
              Dovecot should be outputing info about autoindexing given
              your setup. Does it do that or does it give some other
              message? Can you show those logs?<br>
            </p>
            <p>John<br>
            </p>
          </div>
        </blockquote>
        <p>Larry</p>
        <p>Did you notice any difference between the logging for auto
          indexing and the logging for indexing that you triggered
          manually? Would you mind posting the auto indexing logging for
          a message to that same user (ler)?<br>
        </p>
        <p>best regards</p>
        <p>John<br>
        </p>
      </div>
    </blockquote>
    <p>ok, the hypothesis I'm trying to confirm is that auto indexing is
      taking place under the full username ler@somedomain whereas in the
      command line tests you did your are using user ler (as presumably
      is roundcube).</p>
    <p>John<br>
    </p>
  </div>

</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr">Larry Rosenman                     <a href="http://www.lerctr.org/~ler" target="_blank">http://www.lerctr.org/~ler</a><br><span>Phone: <span title="Call with Google Voice"><span title="Call with Google Voice"><span title="Call with Google Voice">+1 214-642-9640</span></span></span> (c)     E-Mail: </span><a href="mailto:larryrtx@gmail.com" target="_blank">larryrtx@gmail.com</a><br>US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106</div></div></div></div></div></div></div></div>