<!doctype html>
<html>
<head>
<meta charset="UTF-8">
</head>
<body>
<div>
Hi!
</div>
<div>
<br>
</div>
<div>
Thank you for reporting these. We will look into them. In the mean time, can you see if any of these are fixed in 2.3.6?
</div>
<div>
<br>
</div>
<div>
Aki
</div>
<blockquote type="cite">
<div>
On 07/07/2019 18:12 Arnold Opio Oree via dovecot <dovecot@dovecot.org> wrote:
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Dovecot Team,
</div>
<div>
<br>
</div>
<div>
I'd like to report a number of bugs, that are to my view all critical.
</div>
<div>
<br>
</div>
<div>
System: Replicated on multiple Debian 10 (Buster) systems
</div>
<div>
Dovecot Version(s): 2.3.4.1
</div>
<div>
<br>
</div>
<div>
doveadm-sync -1/general
</div>
<div>
<br>
</div>
<div>
1) If DIRNAMEs are not different between command line and mail_location doveadm sync will fail, saying that the source and destination directories are the same
</div>
<div>
<br>
</div>
<div>
2) The -n / -N flags do not work, and a sync will fail strangely if location is specified in the namespace definition
</div>
<div>
<br>
</div>
<div>
3) Adds mbox to path name under mailbox directory (where syncing from an mbox source)
</div>
<div>
<br>
</div>
<div>
4) Not having the mailboxes at source named the same as those at destination causes errors and partial sync
</div>
<div>
<br>
</div>
<div>
5) Not having the target mailboxes formatted to receive the sync (/<mailboxroot>/DIRNAME/) will cause sync errors.
</div>
<div>
<br>
</div>
<div>
doveadm-sync
</div>
<div>
<br>
</div>
<div>
1) With large synchronizations UIDs are corrupted where multiple syncs are executed and the program can no longer synchronize
</div>
<div>
<br>
</div>
<div>
dovecot
</div>
<div>
<br>
</div>
<div>
1) Panics and fails to expand ~ to user home: observed cases are where multiple namespaces are being used
</div>
<div>
<br>
</div>
<div>
Please let me know if you need me to elaborate or to provide any further information that you may need to replicate the bugs, or if I can help in any other way.
</div>
<div>
<br>
</div>
<div>
With regards to the last error that I requested help on i.e. \Noselect. This has been resolved more-or-less by the workarounds that I have implemented for the bugs reported above.
</div>
<div>
<br>
</div>
<div>
I have seen a number of threads whilst researching the \Noselect issue where people have been very confused. My finding was that \Noselect is a function of the IMAP specification server-side implementation RFC3501 (
<a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/2/cturGvUAbhIkMPkkWYpZGw/aHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL3JmYzM1MDEjc2VjdGlvbi02LjMu#section-6.3.">https://tools.ietf.org/html/rfc3501#section-6.3.</a>6). And for me the server was returning directories with \Noselect because the mailboxes were malformed on account of dovadm-sync errors. In order to fix this I formed a bash command to transverse the mailbox hierarchy and create the missing folders critical to the sdbox format, namely DIRNAME.
</div>
<div>
<br>
</div>
<div>
Kind regards,
</div>
<div>
<br>
</div>
<div>
Arnold Opio Oree
</div>
<div>
<div>
Chief Executive Officer
</div>
<div>
Parallax Digital Technologies
</div>
<div>
<br>
</div>
<div>
<a href="mailto:arnoldoree@parallaxdt.com"><strong><span style="color: #850404;">arnoldoree@parallaxdt.com</span></strong></a>
</div>
<div>
<br>
</div>
<div>
<a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/3/oKwvqwVipSqwvpqdaumlnA/aHR0cDovL3d3dy5wYXJhbGxheGR0LmNvbQ"><strong><span style="color: #850404;">http://www.parallaxdt.com</span></strong></a>
</div>
<div>
<br>
</div>
<div>
<strong>tel :</strong> +44 (0) 333 577 8587
</div>
<div>
<strong>fax :</strong> +44 (0) 20 8711 2477
</div>
<div>
<br>
</div>
<div>
<span style="font-size: small;">Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. <span style="color: #850404;">08836288</span></span>
</div>
<div>
<br>
</div>
<div>
<span style="color: #c0c0c0; font-size: xx-small;">The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other </span>persons.
</div>
</div>
<div>
<br>
</div>
<div>
-----Original Message-----
</div>
<div>
<strong>From</strong>: Arnold Opio Oree via dovecot <
<a href="mailto:Arnold%20Opio%20Oree%20via%20dovecot%20%3cdovecot@dovecot.org%3e">dovecot@dovecot.org</a>>
</div>
<div>
<strong>Reply-To</strong>:
<a href="mailto:arnoldoree@parallaxict.com">arnoldoree@parallaxict.com</a>, Arnold Opio Oree <
<a href="mailto:Arnold%20Opio%20Oree%20%3carnold.oree@parallaxict.com%3e">arnold.oree@parallaxict.com</a>>
</div>
<div>
<strong>To</strong>:
<a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>
</div>
<div>
<strong>Cc</strong>:
<a href="mailto:rs@sys4.de">rs@sys4.de</a>,
<a href="mailto:aki.tuomi@open-xchange.com">aki.tuomi@open-xchange.com</a>
</div>
<div>
<strong>Subject</strong>: Re: Applying Dovecot for a large / deep folder-hierarchy archive.
</div>
<div>
<strong>Date</strong>: Thu, 04 Jul 2019 14:52:28 +0100
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
Hi all,
</div>
<div>
<br>
</div>
<div>
The guidance provided so far has been really helpful, and has helped a great deal to bringing down wasted energy on finding and executing a viable path. I am now at the final due action to complete our Dovecot application to our use-case, but am stuck on an issue that I cannot find any easily accessible documentation on.
</div>
<div>
<br>
</div>
<div>
Generally this is what has been done:
</div>
<div>
<br>
</div>
<div>
1. Uploaded the enterprise data PST to the target groupware server.
</div>
<div>
2. Prepared the server by changing the mailbox format to sdbox and the the Dovecot mail location to mail_location=/var/vmail/domain/user/mail/
</div>
<div>
3. Converted the pst (on-server) to a recursive mbox hierarchy using readpst
</div>
<div>
4. Executed doveadm-sync to convert mbox hierarchy data into sdbox and to copy it into the enterprise archive user's mailboxes
</div>
<div>
4.i. The biggest issue I faced at this point was doveadm-sync saying that the source and destination pointed to the same location, whereas they clearly did not.
</div>
<div>
4.i.a. I resolved this by removing the location= setting from the target namespace, and allowing it to default to mail_location = setting, and then using a completely different DIRNAME for the import doveadm-sync execution (which was the desired final DIRNAME); I then once the sync had been successful, changed the mail_location DIRNAME so that it pointed to the imported mail DIRNAME; and hence the imported email data was in the live mailboxes
</div>
<div>
4.i.b. doveadm-import failed several times, and was throwing quite inexplicable errors, so I moved onto doveadm-sync
</div>
<div>
4.i.c. I also had to make sure that the source and destination folder names matched, otherwise doveadm-syc threw very many errors and only partially imported the data
</div>
<div>
4.i.d. An issue which I decided just to live with is that an mbox DIRNAME was added to each mailbox as well as the DIRNAME specified so the path to mail is mbox/dbox-Mails. My thought is that with the data live on an IMAP server it will be possible to do a dysync through TCP to correct this problem.
</div>
<div>
<br>
</div>
<div>
The final issue that I am facing now, is that when readpst finds empty folders in the source pst hierarchy, it does not create an mbox file in the mbox hierarchy folder space. This causes doveadm-sync to not create the target data required for its mailbox structure i.e. DIRNAME sub-folder and index file (with our configuration). At this point either doveadm-sync or the dovecot process makes these empty folders not selectable.
</div>
<div>
<br>
</div>
<div>
The question now is how would I go about making all of these folders selectable, e.g. with an internal or external command line tool to change flags / create necessary sdbox mailbox constituent data?
</div>
<div>
<br>
</div>
<div>
<div>
Many thanks,
</div>
<div>
<br>
</div>
<div>
Arnold Opio Oree
</div>
<div>
Chief Executive Officer
</div>
<div>
Parallax Digital Technologies
</div>
<div>
<br>
</div>
<div>
<a href="mailto:arnoldoree@parallaxdt.com"><strong><span style="color: #850404;">arnoldoree@parallaxdt.com</span></strong></a>
</div>
<div>
<br>
</div>
<div>
<a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/4/UcYJWOlfpjTcpmsYQnA3HA/aHR0cDovL2g1bG0ubWp0Lmx1L2xuay9BTUlBQUVMcVhrb0FBQUFBQUFBQUFLQUZ5blFBQVNLQV9BVUFBQUFBQUFmLXRBQmRIZ1NkTnNtVUpISzRRdVctVVp0WndhSmdYQUFIbTVJLzEvVGtYNTI4RXZyd3p3S04yYUtHN1dGUS9hSFIwY0RvdkwzZDNkeTV3WVhKaGJHeGhlR1IwTG1OdmJR"><strong><span style="color: #850404;">http://www.parallaxdt.com</span></strong></a>
</div>
<div>
<br>
</div>
<div>
<strong>tel :</strong> +44 (0) 333 577 8587
</div>
<div>
<strong>fax :</strong> +44 (0) 20 8711 2477
</div>
<div>
<br>
</div>
<div>
<span style="font-size: small;">Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. <span style="color: #850404;">08836288</span></span>
</div>
<div>
<br>
</div>
<div>
<span style="color: #c0c0c0; font-size: xx-small;">The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other </span>persons.
</div>
</div>
<div>
<br>
</div>
<div>
-----Original Message-----
</div>
<div>
<strong>From</strong>: Arnold Opio Oree via dovecot <
<a href="mailto:Arnold%20Opio%20Oree%20via%20dovecot%20%3cdovecot@dovecot.org%3e">dovecot@dovecot.org</a>>
</div>
<div>
<strong>Reply-To</strong>:
<a href="mailto:arnoldoree@parallaxict.com">arnoldoree@parallaxict.com</a>, Arnold Opio Oree <
<a href="mailto:Arnold%20Opio%20Oree%20%3carnold.oree@parallaxict.com%3e">arnold.oree@parallaxict.com</a>>
</div>
<div>
<strong>To</strong>: Robert Schetterer <
<a href="mailto:Robert%20Schetterer%20%3crs@sys4.de%3e">rs@sys4.de</a>>,
<a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a>
</div>
<div>
<strong>Subject</strong>: Re: Applying Dovecot for a large / deep folder-hierarchy archive.
</div>
<div>
<strong>Date</strong>: Thu, 27 Jun 2019 12:05:35 +0100
</div>
<div>
<br>
</div>
<blockquote type="cite">
<div>
Also you may run into client limits.... i ve seen this with outlook,
</div>
<div>
apple mail, thunderbird via imap in the past
</div>
</blockquote>
<div>
<br>
</div>
<div>
Thanks for this note Robert, it was not really an aspect that I was
</div>
<div>
considering.
</div>
<div>
<br>
</div>
<div>
We are operating our groupware services user access through both
</div>
<div>
Evolution Groupware and KDE Kontact / KMail on Debian Linux
</div>
<div>
workstations. Hopefully if there is a client issue it should be local
</div>
<div>
to only one groupware client.
</div>
<div>
<br>
</div>
<div>
I will be sure to study / investigate in this - client - area should any issues that are not traceable to the server-side arise.
</div>
<div>
<br>
</div>
<div>
Many thanks,
</div>
<div>
<br>
</div>
<div>
Arnold Opio Oree
</div>
<div>
Chief Executive Officer
</div>
<div>
Parallax Digital Technologies
</div>
<div>
<br>
</div>
<div>
<pre>arnoldoree@parallaxdt.com</pre>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<pre>http://www.parallaxdt.com</pre>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
tel : +44 (0) 333 577 8587
</div>
<div>
fax : +44 (0) 20 8711 2477
</div>
<div>
<br>
</div>
<div>
Parallax Digital Technologies is a trading name of Parallax Global
</div>
<div>
Limited. U.K. Co. No. 08836288
</div>
<div>
<br>
</div>
<div>
The contents of this e-mail are confidential. If you are not the
</div>
<div>
intended recipient you are to delete this e-mail immediately, disregard
</div>
<div>
its contents and disclose them to no other persons.
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
-----Original Message-----
</div>
<div>
From: Robert Schetterer via dovecot <
</div>
<div>
<pre>dovecot@dovecot.org</pre>
</div>
<div>
>
</div>
<div>
Reply-To: Robert Schetterer <
</div>
<div>
<pre>rs@sys4.de</pre>
</div>
<div>
>
</div>
<div>
To:
</div>
<div>
<pre>dovecot@dovecot.org</pre>
</div>
<div>
<br>
</div>
<div>
Subject: Re: Applying Dovecot for a large / deep folder-hierarchy
</div>
<div>
archive.
</div>
<div>
Date: Thu, 27 Jun 2019 12:53:49 +0200
</div>
<div>
<br>
</div>
<div>
Am 27.06.2019 um 07:35 schrieb Aki Tuomi via dovecot:
</div>
<blockquote type="cite">
<div>
On 26.6.2019 22.12, Arnold Opio Oree via dovecot wrote:
</div>
<blockquote type="cite">
<div>
Hello to you all,
</div>
<div>
<br>
</div>
<div>
I'd like to ask about my intended application of Dovecot to create
</div>
<div>
a folder-hierarchy for storing our enterprise emails, which are
</div>
<div>
treated as live data rather than archives for compliance or
</div>
<div>
occasional / reactive retrieval.
</div>
<div>
<br>
</div>
<div>
The data is presently not that large (a few gigabytes), but it is
</div>
<div>
expected to grow rapidly. Up to this stage the data has been
</div>
<div>
contained
</div>
<div>
in a Microsoft Exchange mailbox (2013), and then in an offline PST.
</div>
<div>
The move to the offline PST was by necessity, as the large number
</div>
<div>
of
</div>
<div>
folders, and depth of hierarchy to my best understanding caused the
</div>
<div>
exchange server / outlook / evolution mail clients to begin to
</div>
<div>
malfunction. To cope with this the archive was broken up and the
</div>
<div>
bulk stored in the offline PST and the most active components
</div>
<div>
stored in
</div>
<div>
online Exchange mailboxes.
</div>
<div>
<br>
</div>
<div>
I have some understanding of the fs mbox format, and also the
</div>
<div>
mitigations to be made for certain characters / strings. My main
</div>
<div>
concern is whether Dovecot is likely to be able to cope well with a
</div>
<div>
large number of folders / depth of hierarcy.
</div>
<div>
<br>
</div>
<div>
I will really appreciate any help / advice you can give.
</div>
<div>
<br>
</div>
<div>
Best regards,
</div>
<div>
<br>
</div>
<div>
Arnold Opio Oree
</div>
<div>
<br>
</div>
</blockquote>
<div>
<br>
</div>
<div>
Hi!
</div>
<div>
<br>
</div>
<div>
Dovecot 2.2.34/2.3 supports unlimited folder depth, the only limiting
</div>
<div>
factor is that the total name may not exceed 4096 bytes. Also
</div>
<div>
individual
</div>
<div>
folder names may not exceed 255 bytes.
</div>
<div>
<br>
</div>
<div>
Prior to that the limit is 255 per folder up to 16 levels.
</div>
<div>
<br>
</div>
<div>
I can't recommend using 'mbox' storage format, please consider using
</div>
<div>
maildir or sdbox instead.
</div>
<div>
<br>
</div>
<div>
Aki
</div>
<div>
<br>
</div>
</blockquote>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<div>
<br>
</div>
<br>
<img style="height: 1px; width: 1px; border: 0;" border="0" alt="" width="1" height="1" src="http://h5lm.mjt.lu/oo/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/4f4e16d8/e.gif">
</blockquote>
<div>
<br>
</div>
<div class="io-ox-signature">
<pre>---
Aki Tuomi</pre>
</div>
</body>
</html>