Fwd: SOLR/Index?

15 Apr 2019 · *ARE THE SAME MAILBOX*


      forgot to reply all.
---------- Forwarded message ---------
From: Larry Rosenman <larryrtx@gmail.com>
Date: Mon, Apr 15, 2019 at 5:58 AM
Subject: Re: SOLR/Index?
To: John Fawcett <john@voipsupport.it>
the local users (myself, my wife, a friend) can authenticate EITHER as
<username> or <username>@lerctr.org.
switching to all virtual users is NOT going to happen.
If I login to roundcube with <user>@lerctr.org it finds the
autoindexed mail.
So, if I make everyone always authenticate as <user>@lerctr.org we should
be fine.
and change my script to do doveadm -u <user>@<domain> instead of depending
on the local user running the cron job.
question: Is there some way to have dovecot change what it sees to be
<user>@lerctr.org when they login as <user>?
On Mon, Apr 15, 2019 at 5:54 AM John Fawcett via dovecot <
dovecot@dovecot.org> wrote:
...
On 15/04/2019 11:38, Larry Rosenman via dovecot wrote:
⌂63% [ler@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX
⌂67% [ler@thebighonker.lerctr.org:~] 1 $ mail -s "test fts1970"
ler@lerctr.org
test fts1970
test fts1970
.
EOT
[ler@thebighonker.lerctr.org:~] $ mailq
[ler@thebighonker.lerctr.org:~] $ grep fts1970 mail/INBOX
Subject: test fts1970
test fts1970
test fts1970
Apr 15 04:29:03 thebighonker exim[49528]: 1hFxvD-000Csq-P6 <=
ler@lerctr.org U=ler P=local S=388
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Connect from local
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(ler@lerctr.org/49364):
save: box=INBOX, uid=175402, msgid=<
E1hFxvD-000Csq-P6@thebighonker.lerctr.org>, size=640, vsize=660,
from=Larry Rosenman <ler@lerctr.org>, subject=test fts1970, flags=()
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(ler@lerctr.org/49364):
sieve: msgid=<E1hFxvD-000Csq-P6@thebighonker.lerctr.org>: stored mail
into mailbox 'INBOX' (subject=test fts1970 from=ler@lerctr.org size=660)
Apr 15 04:29:03 thebighonker dovecot[2507]: lmtp(49364): Disconnect from
local: Client has quit the connection (state=READY)
Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 => ler <
ler@lerctr.org> R=localuser T=dovecot_lmtp S=404 C="250 2.0.0 <
ler@lerctr.org> 6ACWMN9OtFzUwAAAu+mOrA Saved" QT=0s DT=0s
Apr 15 04:29:03 thebighonker exim[49535]: 1hFxvD-000Csq-P6 Completed QT=0s
Apr 15 04:29:03 thebighonker dovecot[2507]: indexer-worker(
ler@lerctr.org/49366): Indexed 1 messages in INBOX (UIDs 175402..175402)
⌂81% [ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX  body
'fts1970'
⌂83% [ler@thebighonker.lerctr.org:~] $
⌂65% [ler@thebighonker.lerctr.org:~] 75 $ doveadm search -u ler@lerctr.org
mailbox INBOX body 'fts1970'
a53a143be44bda5bd4830000bbe98eac 175402
[ler@thebighonker.lerctr.org:~] $ doveadm index -q INBOX
[ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX  body
'fts1970'
[ler@thebighonker.lerctr.org:~] $ doveadm fts rescan
[ler@thebighonker.lerctr.org:~] $ doveadm index -q INBOX
[ler@thebighonker.lerctr.org:~] $ doveadm search mailbox INBOX  body
'fts1970'
a53a143be44bda5bd4830000bbe98eac 175402
[ler@thebighonker.lerctr.org:~] $ doveadm search -u ler@lerctr.org
mailbox INBOX body 'fts1970'
a53a143be44bda5bd4830000bbe98eac 175402
[ler@thebighonker.lerctr.org:~] $
So, yes, your hypothesis is correct.
Question: How can I make it consistent?
I have a script that runs on the first of the month that does archiving,
and I have similar issues in that namespace:
⌂67% [ler@thebighonker.lerctr.org:~] $ cat bin/archive-mail
#!/bin/sh
PATH=$PATH:/usr/local/bin
#Expects to be run after midnight on the first of the month
#  to archive all the previous months mail
#Date Run:
TODAY=date "+%Y-%m-%d"
#last month in YYYY/MM
YEAR_LAST_MONTH=date -v-1d "+%Y/%m"
#1st of last month as 01-Mon-YYYY
FIRST_LAST_MONTH=date -v-1d "+01-%b-%Y"
echo 'TODAY=' ${TODAY}
echo 'YEAR_LAST_MONTH=' ${YEAR_LAST_MONTH}
echo 'FIRST_LAST_MONTH=' ${FIRST_LAST_MONTH}
# get a list of all the mailboxes with at least one real message
doveadm -f tab mailbox status vsize \* 2>/dev/null |
sed -e 1d | sort -k 1,1 |
awk  'BEGIN {FS="\t"} {if ($2 > 0)  print $1}' |
while read i
do
echo date start ${i}
doveadm mailbox create "ARCHIVE/${YEAR_LAST_MONTH}/${i}"
doveadm -f tab mailbox status messages "${i}"
doveadm move "ARCHIVE/${YEAR_LAST_MONTH}/${i}" mailbox 

"${i}" BEFORE ${TODAY} SINCE ${FIRST_LAST_MONTH}
doveadm -f tab mailbox status messages "${i}"
echo date done  ${i}
done
⌂64% [ler@thebighonker.lerctr.org:~] $
The Exim config can be provided as well if necessary.
ler & ler@lerctr.org *ARE THE SAME MAILBOX*
At the moment it looks as though you have two sets of emails indexed in
solr. One is indexed under username (the one you are running mannually and
apparently the one used by roundcube too, but that's to be verified) and
another set being indexed by autoindex = yes option using the full email
address. Once you've got it working as you require, then you may want to
clean out solr and reindex with just one of them just to reduce volumes.
Your setup seems to have a mix of users from mysql and from /etc/passwd.
Not sure if your mysql users are all mapped to real users or they have
their own mailboxes with domain included. Your solution will depend on what
you really need and if the setup is working correctly you may not want to
tweak it too much or other things may start breaking.
The simplest thing that comes to mind (providing your manually indexed
mails show up in roundcube searches) is just to turn off autoindex and
schedule indexing from cron. As things stand I believe autoindex works well
with virtual users username@domain. You'll probably need to start off
with a rescan if you do this.
If you can swith to all virtual users without local users then probably
you can use autoindex, but I can understand that may not be possible. For
sure I'd recommend doing it in a test environment first so you can be sure
of the configuration and then look at migrating your existing mailboxes
over to virtual users.
John
--
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: larryrtx@gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
--
Larry Rosenman                     http://www.lerctr.org/~ler
Phone: +1 214-642-9640 (c)     E-Mail: larryrtx@gmail.com
US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106

Larry Rosenman

Johan Huldtgren

Larry Rosenman

John Fawcett

Larry Rosenman

John Fawcett

Larry Rosenman

John Fawcett

Larry Rosenman

John Fawcett

Tanstaafl

Larry Rosenman

tags

participants (4)