<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
</head>
<body>
<p>Hello John! Thanks for your answer.<br>
</p>
On 12/9/22 12:30, John Stoffel wrote:<br>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home"><br>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">I have a setup on which dovecot is the server in a domain on which mailboxes are >20GB, some of
them are 150GB or more.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
It looks like you're using Maildir format? In any case, please post
more details of your configuration. </pre>
</blockquote>
<p>Sure. It is a simple setup, with Maildir.</p>
<p><font face="monospace"># doveconf -N</font></p>
<p><font face="monospace"><span
style="color:#000000;background-color:#ffffff;">auth_debug =
no
</span><br>
auth_mechanisms = plain login
<br>
auth_verbose = no
<br>
debug_log_path = /var/log/syslog/mail.debug
<br>
disable_plaintext_auth = no
<br>
login_greeting = Hermes.
<br>
mail_debug = yes
<br>
mail_index_rewrite_max_log_bytes = 1 M
<br>
mail_location = maildir:~/Maildir
<br>
namespace inbox {
<br>
inbox = yes
<br>
location = <br>
mailbox Drafts {
<br>
special_use = \Drafts
<br>
}
<br>
mailbox Junk {
<br>
special_use = \Junk
<br>
}
<br>
mailbox Sent {
<br>
special_use = \Sent
<br>
}
<br>
mailbox "Sent Messages" {
<br>
special_use = \Sent
<br>
}
<br>
mailbox Trash {
<br>
special_use = \Trash
<br>
}
<br>
prefix = <br>
}
<br>
passdb {
<br>
driver = pam
<br>
name = <br>
}
<br>
protocols = " imap"
<br>
service auth {
<br>
unix_listener auth-client {
<br>
group = Debian-exim
<br>
mode = 0660
<br>
user = Debian-exim
<br>
}
<br>
}
<br>
service imap {
<br>
vsz_limit = 2 G
<br>
}
<br>
ssl = yes
<br>
ssl_cert = </etc/dovecot/dovecot-cert.pem
<br>
ssl_client_ca_dir = /etc/ssl/certs
<br>
ssl_dh = # hidden, use -P to show it
<br>
ssl_key = # hidden, use -P to show it
<br>
userdb {
<br>
driver = passwd
<br>
name = <br>
}
<br>
verbose_proctitle = yes
<br>
protocol imap {
<br>
mail_max_userip_connections = 20
<br>
}<br>
</font></p>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home">
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">When reconfiguring _some_ mailboxes with Outlook 365 (configuring a previously used mailbox on a
new PC), the process takes too long even if mail visibility on Outlook is 3 months.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
So the outlook client can't download and parse the number of messages
on there without breaking? How many messages are there? It's
probably not the size of the mailbox, but the number of messages.</pre>
</blockquote>
50K mails on some folders... total 5.5M on the whole server. I did
discover which was the problem but I don't have a solution.<br>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home">
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Inspecting at INBOX cur directory I did note that the 'ls' command shows the last file listed is a
file from year 2020, 2019 or whatever not being the last received mail; that means the
alphabetical ordering of 'ls' is showing past years after the last received mail, i.e. dovecot
changed the epoch in filename. This only happens with Outlook. Following is an excerpt of ls:
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
Sure, 'ls -l' doesn't do any sorting, it just reads the directory
information as returned from the disk and show you the results. If
you want it by time, you need to do:
ls -ltr
to have the newest files be at the end. But how 'ls' sees the
directry entries doen't matter to dovecot. </pre>
</blockquote>
Newest files are at the end; 'ls' without '-U' or '--sort=none'
lists files ordered alphabetically, so the last entry would be the
last mail placed on the folder because the epoch time is part of the
filename in Maildir structure...<br>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home">
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">-rw-r--r-- 1 coord.ventas ventas 544765 sep 9 10:57 1662769412.M329925P1259441.xyz.com,S=
544765,W=552045:2,
-rw-r--r-- 1 coord.ventas ventas 163491 sep 5 12:14 1662769412.M74257P1259441.xyz.com,S=
163491,W=165846:2,S
-rw-r--r-- 1 coord.ventas ventas 3043536 sep 9 12:02 1662769412.M777084P1259441.xyz.com,S=
3043536,W=3083246:2,
-rw-r--r-- 1 coord.ventas ventas 161002 feb 27 2020 1662874173.M608062P1282222.xyz.com,S=
161002,W=163373:2,S
-rw-r--r-- 1 coord.ventas ventas 2230491 feb 27 2020 1662874176.M281294P1282222.xyz.com,S=
2230491,W=2259506:2,S
-rw-r--r-- 1 coord.ventas ventas 167925 feb 27 2020 1662874176.M741229P1282222.xyz.com,S=
167925,W=170373:2,S
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">The process of renaming files is continuous until all folder was
reread and renamed every file on it. If I issue 'lsof' (with grep) I
see the file that is being renamed passing through tmp/ directory
and after that, the file is placed under new/ directory (I guess)
and Outlook sees a new email to sync, making the sync process too
long... mails that have been synced are resynced...
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
Wait what? Can you explain exactly what you're doing here? Why would
O365 client be renaming the file </pre>
</blockquote>
<p>Ok, I disabled TLS options and inspected the traffic with
tcpdump. I did note unnecesary STORE, EXPUNGE and APPEND
operations on previously synced folder. I guess the whole mailbox
is analyzed (from the local cache on .pst file) and O365 tags as
SPAM a large number of innocent emails (including local mails from
the same domain which does not travel through internet to reach
the destination).</p>
<p>Also I'm using port 144 instead of 143 to bypass GData antivirus
SPAM checks. Anyway I completely disabled GData antivirus and none
of its services remains running on these workstations; also
starting Outlook with safe mode exhibits this behavior, so I don't
know which feature is this on O365 that rescan all stored emails
but it is beyond the scope of this list because there is no
feature on dovecot that can be enabled to workaround this. I guess
this is a bug on O365.</p>
<p>All mails are retransmitted to the server with the subject
modified (tagging them as SPAM or SPAM suspected) and obviously
dovecot stores them with a new epoch timestamp on filename and
that's why I have mails with older timestamp but with recent epoch
time on filename...<br>
</p>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home">
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">Take a look at the timestamp on filenames: 1662769412 corresponds to
"Fri Sep 9 20:23:32 -04 2022" and 1662874176 corresponds to Sun Sep
11 01:29:36 -04 2022 but mails were neither sent nor received on
that dates.
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
</pre>
<blockquote type="cite">
<pre class="moz-quote-pre" wrap="">What is Outlook requesting imap server to do and how to avoid this?
</pre>
</blockquote>
<pre class="moz-quote-pre" wrap="">
What is the outlook client version? What are your connection
settings? Have you looked in the logs on the dovecot server side to
see what they say?</pre>
</blockquote>
Outlook 365 is version 16.15601.20148 compilation 2208 on one
workstation that exhibits the issue and this same version on another
PC on which this issue is not present... <br>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home">
<pre class="moz-quote-pre" wrap="">
What working clients are your users using today? Are they working
properly with older clients? </pre>
</blockquote>
<p>Basically I'm the mail server administrator not the sysadmin on
the network. All users prefer working with O365. I've installed
Thunderbird on every PC but users refuses to use it in favor of
using Outlook, unfortunately.</p>
<blockquote type="cite"
cite="mid:25375.24229.44271.252498@quad.stoffel.home">
<pre class="moz-quote-pre" wrap="">
Basically, you have to assume we know nothing and explain your setup
and configuration to us like we're idiots who don't know anything.
Spell out all the details please. :-)</pre>
</blockquote>
<p>Yes, I'm sorry. I hope with this mail I've clarified the
scenario.</p>
<p>Thanks again!</p>
<p>Emilio</p>
<br>
</body>
</html>