SOLR/Index?
Larry Rosenman
larryrtx at gmail.com
Mon Apr 15 12:38:16 EEST 2019
⌂63% [ler at thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX
⌂67% [ler at thebighonker.lerctr.org:~] 1 $ mail -s "test fts1970"
ler at lerctr.org
test fts1970
test fts1970
.
EOT
[ler at thebighonker.lerctr.org:~] $ mailq
[ler at thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX
Subject: test fts1970
test fts1970
test fts1970
Apr 15 04:29:03 thebighonker exim[49528]: 1hFxvD-000Csq-P6 <= ler at lerctr.org
U=ler P=local S=388
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Connect from local
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(ler at lerctr.org/49364):
save: box=INBOX, uid=175402, msgid=<
E1hFxvD-000Csq-P6 at thebighonker.lerctr.org>, size=640, vsize=660, from=Larry
Rosenman <ler at lerctr.org>, subject=test fts1970, flags=()
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(ler at lerctr.org/49364):
sieve: msgid=<E1hFxvD-000Csq-P6 at thebighonker.lerctr.org>: stored mail into
mailbox 'INBOX' (subject=test fts1970 from=ler at lerctr.org size=660)
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Disconnect from
local: Client has quit the connection (state=READY)
Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 => ler <
ler at lerctr.org> R=localuser T=dovecot_lmtp S=404 C="250 2.0.0 <
ler at lerctr.org> 6ACWMN9OtFzUwAAAu+mOrA Saved" QT=0s DT=0s
Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 Completed QT=0s
Apr 15 04:29:03 thebighonker dovecot[2507]: indexer-worker(
ler at lerctr.org/49366): Indexed 1 messages in INBOX (UIDs 175402..175402)
⌂81% [ler at thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX body
'fts1970'
⌂83% [ler at thebighonker.lerctr.org:~] $
⌂65% [ler at thebighonker.lerctr.org:~] 75 $ doveadm search -u ler at lerctr.org
mailbox INBOX body 'fts1970'
a53a143be44bda5bd4830000bbe98eac 175402
[ler at thebighonker.lerctr.org:~] $ doveadm index -q INBOX
[ler at thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX body
'fts1970'
[ler at thebighonker.lerctr.org:~] $ doveadm fts rescan
[ler at thebighonker.lerctr.org:~] $ doveadm index -q INBOX
[ler at thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX body
'fts1970'
a53a143be44bda5bd4830000bbe98eac 175402
[ler at thebighonker.lerctr.org:~] $ doveadm search -u ler at lerctr.org mailbox
INBOX body 'fts1970'
a53a143be44bda5bd4830000bbe98eac 175402
[ler at thebighonker.lerctr.org:~] $
So, yes, your hypothesis is correct.
Question: How can I make it consistent?
I have a script that runs on the first of the month that does archiving,
and I have similar issues in that namespace:
⌂67% [ler at thebighonker.lerctr.org:~] $ cat bin/archive-mail
#!/bin/sh
PATH=$PATH:/usr/local/bin
#Expects to be run after midnight on the first of the month
# to archive all the previous months mail
#Date Run:
TODAY=`date "+%Y-%m-%d"`
#last month in YYYY/MM
YEAR_LAST_MONTH=`date -v-1d "+%Y/%m"`
#1st of last month as 01-Mon-YYYY
FIRST_LAST_MONTH=`date -v-1d "+01-%b-%Y"`
echo 'TODAY=' ${TODAY}
echo 'YEAR_LAST_MONTH=' ${YEAR_LAST_MONTH}
echo 'FIRST_LAST_MONTH=' ${FIRST_LAST_MONTH}
# get a list of all the mailboxes with at least one real message
doveadm -f tab mailbox status vsize \* 2>/dev/null |
sed -e 1d | sort -k 1,1 |
awk 'BEGIN {FS="\t"} {if ($2 > 0) print $1}' |
while read i
do
echo `date` start ${i}
doveadm mailbox create "ARCHIVE/${YEAR_LAST_MONTH}/${i}"
doveadm -f tab mailbox status messages "${i}"
doveadm move "ARCHIVE/${YEAR_LAST_MONTH}/${i}" mailbox \
"${i}" BEFORE ${TODAY} SINCE ${FIRST_LAST_MONTH}
doveadm -f tab mailbox status messages "${i}"
echo `date` done ${i}
done
⌂64% [ler at thebighonker.lerctr.org:~] $
The Exim config can be provided as well if necessary.
ler & ler at lerctr.org *ARE THE SAME MAILBOX*
On Mon, Apr 15, 2019 at 4:05 AM John Fawcett via dovecot <
dovecot at dovecot.org> wrote:
> On 15/04/2019 10:59, Larry Rosenman via dovecot wrote:
>
> I'll run a full test when I'm back in front of areal computer vs. My
> phone.(in a few hours)
>
> Get Outlook for Android <https://aka.ms/ghei36>
>
> ------------------------------
> *From:* dovecot <dovecot-bounces at dovecot.org>
> <dovecot-bounces at dovecot.org> on behalf of John Fawcett via dovecot
> <dovecot at dovecot.org> <dovecot at dovecot.org>
> *Sent:* Monday, April 15, 2019 3:57:08 AM
> *To:* Dovecot Mailing List
> *Subject:* Re: SOLR/Index?
>
> On 15/04/2019 10:31, Larry Rosenman via dovecot wrote:
>
> It always shows the autoindex. And yes built from sources. I'm the
> FreeBSD port maintainer for mail/docecot. This has been happening for
> several releases.
>
> Get Outlook for Android <https://aka.ms/ghei36>
>
> ------------------------------
> *From:* dovecot <dovecot-bounces at dovecot.org>
> <dovecot-bounces at dovecot.org> on behalf of John Fawcett via dovecot
> <dovecot at dovecot.org> <dovecot at dovecot.org>
> *Sent:* Monday, April 15, 2019 2:06:55 AM
> *To:* dovecot at dovecot.org
> *Subject:* Re: SOLR/Index?
>
> On 15/04/2019 08:09, Larry Rosenman via dovecot wrote:
>
> Note the hits after the fts rescan/index.
>
> Get Outlook for Android <https://aka.ms/ghei36>
>
> ------------------------------
> *From:* Aki Tuomi <aki.tuomi at open-xchange.com>
> <aki.tuomi at open-xchange.com>
> *Sent:* Monday, April 15, 2019 12:55:07 AM
> *To:* Larry Rosenman; John Fawcett
> *Cc:* Dovecot Mailing List
> *Subject:* Re: SOLR/Index?
>
>
>
> On 15.4.2019 3.33, Larry Rosenman via dovecot wrote:
>
> ⌂72% [ler at thebighonker.lerctr.org:~] $ doveadm search mailbox
> lists/freebsd/ports-commiters body 'sysutils'
> [ler at thebighonker.lerctr.org:~] $ doveadm fts rescan
> [ler at thebighonker.lerctr.org:~] $ doveadm index -q
> lists/freebsd/ports-commiters
> ⌂64% [ler at thebighonker.lerctr.org:~] $ tail -f /var/log/maillog
> Apr 14 19:30:27 thebighonker dovecot[2507]: imap-login: Disconnected (auth
> failed, 1 attempts in 2 secs): user=<gwc>, method=PLAIN,
> rip=180.180.217.124, lip=192.147.25.65, TLS: Connection closed,
> session=<EAk5woaGV4S0tNl8>
> Apr 14 19:30:28 thebighonker dovecot[2507]: imap-login: Login: user=<ler>,
> method=PLAIN, rip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
> lip=2001:470:1f0f:3ad:bb:dcff:fe50:d900, mpid=14813, TLS,
> session=<lr1mwoaGwYMgAQRwHw8DrQC73P/+UNkA>
> Apr 14 19:30:30 thebighonker dovecot[2507]: imap(ler/14813): Logged out
> in=12412 out=66691 fhc=0 fhb=0 fbc=0 fbb=0 del=0 exp=0 trash=0
> Apr 14 19:30:54 thebighonker exim[14846]: no host name found for IP
> address 23.100.68.192
> Apr 14 19:30:55 thebighonker exim[14846]: H=(
> DaVinci-MWare.prophet21lab.com) [23.100.68.192]:52130
> I=[192.147.25.65]:25 sender verify defer for <info at duke.org>: host lookup
> did not complete
> Apr 14 19:30:55 thebighonker exim[14846]: H=(
> DaVinci-MWare.prophet21lab.com) [23.100.68.192]:52130
> I=[192.147.25.65]:25 F=<info at duke.org> temporarily rejected RCPT <
> jpotyh at why.net>: Could not complete sender verify
> Apr 14 19:31:04 thebighonker dovecot[2507]: imap-login: Login: user=<ctr>,
> method=PLAIN, rip=2001:470:1f0f:3ad:bb:dcff:fe50:d900,
> lip=2001:470:1f0f:3ad:bb:dcff:fe50:d900, mpid=14910, TLS,
> session=<bVWMxIaGJtogAQRwHw8DrQC73P/+UNkA>
> Apr 14 19:31:04 thebighonker dovecot[2507]: imap(ctr/14910): Logged out
> in=169 out=1711 fhc=0 fhb=0 fbc=0 fbb=0 del=0 exp=0 trash=0
> Apr 14 19:31:16 thebighonker exim[14911]: no host name found for IP
> address 80.253.235.35
> Apr 14 19:31:19 thebighonker dovecot[2507]: indexer-worker(ler/14919):
> Indexed 1578 messages in lists/freebsd/ports-commiters (UIDs 21067..22644)
> ^C
> [ler at thebighonker.lerctr.org:~] 130 $ doveadm search mailbox
> lists/freebsd/ports-commiters body 'sysutils/'
>
>
> Just minor nit, but you are searching for 'sysutils' first, then
> 'sysutils/'. FTS does not do substring searches by default.
>
> Aki
>
>
> --
> Larry Rosenman http://www.lerctr.org/~ler
> Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
>
> Larry
>
> just to be sure: are you running a standard unmodified 2.3.5.1 version
> which you built from source code?
>
> I can see that first you search for sysutils, then do a rescan and reindex
> (which is shown in the log) and then you are able to find sysutils/.
>
> It is better when doing these tests to search for the same string before
> and after, just to eliminate too many different factors in the test.
>
> Nevertheless I did not see your logging for what happens when you receive
> a test message containing sysutils/. Dovecot should be outputing info about
> autoindexing given your setup. Does it do that or does it give some other
> message? Can you show those logs?
>
> John
>
> Larry
>
> Did you notice any difference between the logging for auto indexing and
> the logging for indexing that you triggered manually? Would you mind
> posting the auto indexing logging for a message to that same user (ler)?
>
> best regards
>
> John
>
> ok, the hypothesis I'm trying to confirm is that auto indexing is taking
> place under the full username ler at somedomain whereas in the command line
> tests you did your are using user ler (as presumably is roundcube).
>
> John
>
--
Larry Rosenman http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c) E-Mail: larryrtx at gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://dovecot.org/pipermail/dovecot/attachments/20190415/5ee606d0/attachment-0001.html>
More information about the dovecot
mailing list