<html dir="ltr"><head></head><body style="text-align: left; direction: ltr; word-wrap: break-word; -webkit-nbsp-mode: space; line-break: after-white-space;"><div>Dovecot Team,</div><div><br></div><div>I'd like to report a number of bugs, that are to my view all critical.</div><div><br></div><div>System: Replicated on multiple Debian 10 (Buster) systems</div><div>Dovecot Version(s): 2.3.4.1</div><div><br></div><div>doveadm-sync -1/general</div><div><br></div><div>1) If DIRNAMEs are not different between command line and mail_location doveadm sync will fail, saying that the source and destination directories are the same</div><div><br></div><div>2) The -n / -N flags do not work, and a sync will fail strangely if location is specified in the namespace definition</div><div><br></div><div>3) Adds mbox to path name under mailbox directory (where syncing from an mbox source)</div><div><br></div><div>4) Not having the mailboxes at source named the same as those at destination causes errors and partial sync </div><div><br></div><div>5) Not having the target mailboxes formatted to receive the sync (/<mailboxroot>/DIRNAME/) will cause sync errors.</div><div><br></div><div>doveadm-sync</div><div><br></div><div>1) With large synchronizations UIDs are corrupted where multiple syncs are executed and the program can no longer synchronize</div><div><br></div><div>dovecot</div><div><br></div><div>1) Panics and fails to expand ~ to user home: observed cases are where multiple namespaces are being used</div><div><br></div><div>Please let me know if you need me to elaborate or to provide any further information that you may need to replicate the bugs, or if I can help in any other way.</div><div><br></div><div>With regards to the last error that I requested help on i.e. \Noselect. This has been resolved more-or-less by the workarounds that I have implemented for the bugs reported above.</div><div><br></div><div>I have seen a number of threads whilst researching the \Noselect issue where people have been very confused. My finding was that \Noselect is a function of the IMAP specification server-side implementation RFC3501 (<a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/1/nJ1qwIFgSyW4nfuZp-VMgA/aHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL3JmYzM1MDEjc2VjdGlvbi02LjMuNg#section-6.3.6"></a><a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/2/cturGvUAbhIkMPkkWYpZGw/aHR0cHM6Ly90b29scy5pZXRmLm9yZy9odG1sL3JmYzM1MDEjc2VjdGlvbi02LjMu#section-6.3.">https://tools.ietf.org/html/rfc3501#section-6.3.</a>6). And for me the server was returning directories with \Noselect because the mailboxes were malformed on account of dovadm-sync errors. In order to fix this I formed a bash command to transverse the mailbox hierarchy and create the missing folders critical to the sdbox format, namely DIRNAME.</div><div><br></div><div>Kind regards,</div><div><br></div><div>Arnold Opio Oree</div><div><span class="-x-evo-signature" id="c44a6ef516cda7a3ca266d6dcf547d8dfc1222b6"><div>Chief Executive Officer</div><div>Parallax Digital Technologies</div><div><br></div><div><a href="mailto:arnoldoree@parallaxdt.com"><b><font color="#850404">arnoldoree@parallaxdt.com</font></b></a></div><div><br></div><div><a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/3/oKwvqwVipSqwvpqdaumlnA/aHR0cDovL3d3dy5wYXJhbGxheGR0LmNvbQ"><b><font color="#850404">http://www.parallaxdt.com</font></b></a></div><div><br></div><div><b>tel :</b> +44 (0) 333 577 8587</div><div><b>fax :</b> +44 (0) 20 8711 2477</div><div><br></div><div><font size="2">Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. <font color="#850404">08836288</font></font></div><div><br></div><div><font size="1" color="#c0c0c0">The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other </font>persons.</div></span></div><div><br></div><div>-----Original Message-----</div><div><b>From</b>: Arnold Opio Oree via dovecot <<a href="mailto:Arnold%20Opio%20Oree%20via%20dovecot%20%3cdovecot@dovecot.org%3e">dovecot@dovecot.org</a>></div><div><b>Reply-To</b>: <a href="mailto:arnoldoree@parallaxict.com">arnoldoree@parallaxict.com</a>, Arnold Opio Oree <<a href="mailto:Arnold%20Opio%20Oree%20%3carnold.oree@parallaxict.com%3e">arnold.oree@parallaxict.com</a>></div><div><b>To</b>: <a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a></div><div><b>Cc</b>: <a href="mailto:rs@sys4.de">rs@sys4.de</a>, <a href="mailto:aki.tuomi@open-xchange.com">aki.tuomi@open-xchange.com</a></div><div><b>Subject</b>: Re: Applying Dovecot for a large / deep folder-hierarchy archive.</div><div><b>Date</b>: Thu, 04 Jul 2019 14:52:28 +0100</div><div><br></div><div><br></div><div>Hi all,</div><div><br></div><div>The guidance provided so far has been really helpful, and has helped a great deal to bringing down wasted energy on finding and executing a viable path. I am now at the final due action to complete our Dovecot application to our use-case, but am stuck on an issue that I cannot find any easily accessible documentation on.</div><div><br></div><div>Generally this is what has been done:</div><div><br></div><div>1. Uploaded the enterprise data PST to the target groupware server.</div><div>2. Prepared the server by changing the mailbox format to sdbox and the the Dovecot mail location to mail_location=/var/vmail/domain/user/mail/</div><div>3. Converted the pst (on-server) to a recursive mbox hierarchy using readpst</div><div>4. Executed doveadm-sync to convert mbox hierarchy data into sdbox and to copy it into the enterprise archive user's mailboxes</div><div>4.i. The biggest issue I faced at this point was doveadm-sync saying that the source and destination pointed to the same location, whereas they clearly did not. </div><div>4.i.a. I resolved this by removing the location= setting from the target namespace, and allowing it to default to mail_location = setting, and then using a completely different DIRNAME for the import doveadm-sync execution (which was the desired final DIRNAME); I then once the sync had been successful, changed the mail_location DIRNAME so that it pointed to the imported mail DIRNAME; and hence the imported email data was in the live mailboxes</div><div>4.i.b. doveadm-import failed several times, and was throwing quite inexplicable errors, so I moved onto doveadm-sync</div><div>4.i.c. I also had to make sure that the source and destination folder names matched, otherwise doveadm-syc threw very many errors and only partially imported the data</div><div>4.i.d. An issue which I decided just to live with is that an mbox DIRNAME was added to each mailbox as well as the DIRNAME specified so the path to mail is mbox/dbox-Mails. My thought is that with the data live on an IMAP server it will be possible to do a dysync through TCP to correct this problem.</div><div><br></div><div>The final issue that I am facing now, is that when readpst finds empty folders in the source pst hierarchy, it does not create an mbox file in the mbox hierarchy folder space. This causes doveadm-sync to not create the target data required for its mailbox structure i.e. DIRNAME sub-folder and index file (with our configuration). At this point either doveadm-sync or the dovecot process makes these empty folders not selectable.</div><div><br></div><div>The question now is how would I go about making all of these folders selectable, e.g. with an internal or external command line tool to change flags / create necessary sdbox mailbox constituent data?</div><div><br></div><div><span><div>Many thanks,</div><div><br></div><div>Arnold Opio Oree</div><div>Chief Executive Officer</div><div>Parallax Digital Technologies</div><div><br></div><div><a href="mailto:arnoldoree@parallaxdt.com"><b><font color="#850404">arnoldoree@parallaxdt.com</font></b></a></div><div><br></div><div><a href="http://h5lm.mjt.lu/lnk/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/4/UcYJWOlfpjTcpmsYQnA3HA/aHR0cDovL2g1bG0ubWp0Lmx1L2xuay9BTUlBQUVMcVhrb0FBQUFBQUFBQUFLQUZ5blFBQVNLQV9BVUFBQUFBQUFmLXRBQmRIZ1NkTnNtVUpISzRRdVctVVp0WndhSmdYQUFIbTVJLzEvVGtYNTI4RXZyd3p3S04yYUtHN1dGUS9hSFIwY0RvdkwzZDNkeTV3WVhKaGJHeGhlR1IwTG1OdmJR"><b><font color="#850404">http://www.parallaxdt.com</font></b></a></div><div><br></div><div><b>tel :</b> +44 (0) 333 577 8587</div><div><b>fax :</b> +44 (0) 20 8711 2477</div><div><br></div><div><font size="2">Parallax Digital Technologies is a trading name of Parallax Global Limited. U.K. Co. No. <font color="#850404">08836288</font></font></div><div><br></div><div><font size="1" color="#c0c0c0">The contents of this e-mail are confidential. If you are not the intended recipient you are to delete this e-mail immediately, disregard its contents and disclose them to no other </font>persons.</div></span></div><div><br></div><div>-----Original Message-----</div><div><b>From</b>: Arnold Opio Oree via dovecot <<a href="mailto:Arnold%20Opio%20Oree%20via%20dovecot%20%3cdovecot@dovecot.org%3e">dovecot@dovecot.org</a>></div><div><b>Reply-To</b>: <a href="mailto:arnoldoree@parallaxict.com">arnoldoree@parallaxict.com</a>, Arnold Opio Oree <<a href="mailto:Arnold%20Opio%20Oree%20%3carnold.oree@parallaxict.com%3e">arnold.oree@parallaxict.com</a>></div><div><b>To</b>: Robert Schetterer <<a href="mailto:Robert%20Schetterer%20%3crs@sys4.de%3e">rs@sys4.de</a>>, <a href="mailto:dovecot@dovecot.org">dovecot@dovecot.org</a></div><div><b>Subject</b>: Re: Applying Dovecot for a large / deep folder-hierarchy archive.</div><div><b>Date</b>: Thu, 27 Jun 2019 12:05:35 +0100</div><div><br></div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div>Also you may run into client limits.... i ve seen this with outlook, </div><div>apple mail, thunderbird via imap in the past</div></blockquote><div><br></div><div>Thanks for this note Robert, it was not really an aspect that I was</div><div>considering. </div><div><br></div><div>We are operating our groupware services user access through both</div><div>Evolution Groupware and KDE Kontact / KMail on Debian Linux</div><div>workstations. Hopefully if there is a client issue it should be local</div><div>to only one groupware client. </div><div><br></div><div>I will be sure to study / investigate in this - client - area should any issues that are not traceable to the server-side arise.</div><div><br></div><div>Many thanks,</div><div><br></div><div>Arnold Opio Oree</div><div>Chief Executive Officer</div><div>Parallax Digital Technologies</div><div><br></div><div><pre>arnoldoree@parallaxdt.com</pre></div><div><br></div><div><br></div><div><pre>http://www.parallaxdt.com</pre></div><div><br></div><div><br></div><div>tel : +44 (0) 333 577 8587</div><div>fax : +44 (0) 20 8711 2477</div><div><br></div><div>Parallax Digital Technologies is a trading name of Parallax Global</div><div>Limited. U.K. Co. No. 08836288</div><div><br></div><div>The contents of this e-mail are confidential. If you are not the</div><div>intended recipient you are to delete this e-mail immediately, disregard</div><div>its contents and disclose them to no other persons.</div><div><br></div><div><br></div><div>-----Original Message-----</div><div>From: Robert Schetterer via dovecot <</div><div><pre>dovecot@dovecot.org</pre></div><div>></div><div>Reply-To: Robert Schetterer <</div><div><pre>rs@sys4.de</pre></div><div>></div><div>To: </div><div><pre>dovecot@dovecot.org</pre></div><div><br></div><div>Subject: Re: Applying Dovecot for a large / deep folder-hierarchy</div><div>archive.</div><div>Date: Thu, 27 Jun 2019 12:53:49 +0200</div><div><br></div><div>Am 27.06.2019 um 07:35 schrieb Aki Tuomi via dovecot:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div>On 26.6.2019 22.12, Arnold Opio Oree via dovecot wrote:</div><blockquote type="cite" style="margin:0 0 0 .8ex; border-left:2px #729fcf solid;padding-left:1ex"><div>Hello to you all,</div><div><br></div><div>I'd like to ask about my intended application of Dovecot to create</div><div>a folder-hierarchy for storing our enterprise emails, which are</div><div>treated as live data rather than archives for compliance or</div><div>occasional / reactive retrieval.</div><div><br></div><div>The data is presently not that large (a few gigabytes), but it is</div><div>expected to grow rapidly. Up to this stage the data has been</div><div>contained</div><div>in a Microsoft Exchange mailbox (2013), and then in an offline PST.</div><div>The move to the offline PST was by necessity, as the large number</div><div>of</div><div>folders, and depth of hierarchy to my best understanding caused the</div><div>exchange server / outlook / evolution mail clients to begin to</div><div>malfunction. To cope with this the archive was broken up and the</div><div>bulk stored in the offline PST and the most active components</div><div>stored in</div><div>online Exchange mailboxes.</div><div><br></div><div>I have some understanding of the fs mbox format, and also the</div><div>mitigations to be made for certain characters / strings. My main</div><div>concern is whether Dovecot is likely to be able to cope well with a</div><div>large number of folders / depth of hierarcy.</div><div><br></div><div>I will really appreciate any help / advice you can give.</div><div><br></div><div>Best regards,</div><div><br></div><div>Arnold Opio Oree</div><div><br></div></blockquote><div><br></div><div>Hi!</div><div><br></div><div>Dovecot 2.2.34/2.3 supports unlimited folder depth, the only limiting</div><div>factor is that the total name may not exceed 4096 bytes. Also</div><div>individual</div><div>folder names may not exceed 255 bytes.</div><div><br></div><div>Prior to that the limit is 255 per folder up to 16 levels.</div><div><br></div><div>I can't recommend using 'mbox' storage format, please consider using</div><div>maildir or sdbox instead.</div><div><br></div><div>Aki</div><div><br></div></blockquote><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div><div><br></div>
<br/><img src="http://h5lm.mjt.lu/oo/AMwAAEPRQ68AAAAAAAAAAKAFynQAASKA_AUAAAAAAAf-tABdIgvzBSx7xYU6SQW2MDXORHSy0wAHm5I/4f4e16d8/e.gif" height="1" width="1" alt="" border="0" style="height:1px;width:1px;border:0;"/>
</body></html>