My sub-folder with Outlook work-around to date
Hi All, please be kind, this is my first e-mail to the list :-)
I actively support CentOS based e-mail servers running Dovecot, Sendmail, Spammassassin and 3 x SOGo based setups.
Dovecot is my goto IMAP server and have used it and modifications to it to net excellent results for years.
Then we have people who insist on only using Outlook (and in some instances the MS Live Messenger thingy).
Some shenanigans in recent years have arose with using sub-folders in the Outlook clients (2013 and 2016).
Traditionally, placing a "/" after the name of a newly desired e-mail folder has netted the result of something like:
"Rentals/"
creating:
/u/home/someuser/mail/Rentals/
Under which users then create actual e-mail folders under the "Rentals" Linux directory as such.
With Outlook 2013 and 2016 this seems to have stopped working and so I implemented a work-around where the user creates a normal folder, example "Rentals++", and I have written a cron script that trawls the $HOMEs each minute and if it finds a folder with a "++" at the end, it creates the folder as a directory, so:
/u/home/someuser/mail/Rentals++
becomes:
/u/home/someuser/mail/Rentals/
and adds this new subscription to their .subscription folder. It also sends an e-mail to the user advising that the new folder is created and they can proceed to use the "Rentals" folder for adding sub-folders (as in real text based mail folders).
The script was a quick work-around one weekend in a mad flurry to get things working and to date works but is not 'user-proof'. So I am now looking at developing an internal web interface to do the same thing and hoping with more controls, has no or far less margin for user error. Long times of Outlook folder refreshes don't help and users sometimes end up with issues that require my Linux command line help.
I have been trawling e-mail forums for some time now and have not seen any other work-arounds (or perhaps I am living under a rock) but before I embark on this web interface adventure, I just wanted to make sure I had not missed some fundamental 'bit' that I should be observing.
All servers are either CentOS 6.9 (or slightly less) and CentOS 7 with the latest updates and for things like Thunderbird and Roundcube and SOGo, work well. I need to experiment with the whole "/" for these but I am currently driven by the enforced Outlook chains.
Any input from you guys on whether this is my best approach or 'hey mate, just do this', would be much appreciated.
I am happy to share my travels script/web-wise if this is the only option to date.
--
As always, I remain at your service.
Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant) E-mail: david@davrom.com
Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies
Hi,
What we do, on the dovecot side, is:
mail_location = maildir:~/Maildir:LAYOUT=fs:DIRNAME=mAildir
See: https://wiki2.dovecot.org/MailLocation/Maildir
This has been working out very nicely for many years.
I'm not sure though that there is a way to 'move' to that config without having to basically migrate all your mailboxes.
MJ
On 10/06/2017 04:08 AM, David.M.Clark wrote:
Hi All, please be kind, this is my first e-mail to the list :-)
I actively support CentOS based e-mail servers running Dovecot, Sendmail, Spammassassin and 3 x SOGo based setups.
Dovecot is my goto IMAP server and have used it and modifications to it to net excellent results for years.
Then we have people who insist on only using Outlook (and in some instances the MS Live Messenger thingy).
Some shenanigans in recent years have arose with using sub-folders in the Outlook clients (2013 and 2016).
Traditionally, placing a "/" after the name of a newly desired e-mail folder has netted the result of something like:
"Rentals/"
creating:
/u/home/someuser/mail/Rentals/
Under which users then create actual e-mail folders under the "Rentals" Linux directory as such.
With Outlook 2013 and 2016 this seems to have stopped working and so I implemented a work-around where the user creates a normal folder, example "Rentals++", and I have written a cron script that trawls the $HOMEs each minute and if it finds a folder with a "++" at the end, it creates the folder as a directory, so:
/u/home/someuser/mail/Rentals++
becomes:
/u/home/someuser/mail/Rentals/
and adds this new subscription to their .subscription folder. It also sends an e-mail to the user advising that the new folder is created and they can proceed to use the "Rentals" folder for adding sub-folders (as in real text based mail folders).
The script was a quick work-around one weekend in a mad flurry to get things working and to date works but is not 'user-proof'. So I am now looking at developing an internal web interface to do the same thing and hoping with more controls, has no or far less margin for user error. Long times of Outlook folder refreshes don't help and users sometimes end up with issues that require my Linux command line help.
I have been trawling e-mail forums for some time now and have not seen any other work-arounds (or perhaps I am living under a rock) but before I embark on this web interface adventure, I just wanted to make sure I had not missed some fundamental 'bit' that I should be observing.
All servers are either CentOS 6.9 (or slightly less) and CentOS 7 with the latest updates and for things like Thunderbird and Roundcube and SOGo, work well. I need to experiment with the whole "/" for these but I am currently driven by the enforced Outlook chains.
Any input from you guys on whether this is my best approach or 'hey mate, just do this', would be much appreciated.
I am happy to share my travels script/web-wise if this is the only option to date.
Thanks for the reply MJ.
Good to hear it works in MailDir format - might need to look at this.
To date I have only had this issue with Outlook and for me personally on Fedora with Thunderbird, I tend to keep a heap of e-mail folders without sub-foldering and have my own little Linux 'utopia' of annually archiving folders etc using the Linux scripting environment, so I never have these issues..... but is a client requirement so will see what I can do from here - they were a bit excited about a web interface to do the "/" bit to get a "master/sub-folder" working so.... I still may implement this if MailDir is too much of a hassle.
Thanks again for your reply.
On 06/10/17 16:42, mj wrote:
Hi,
What we do, on the dovecot side, is:
mail_location = maildir:~/Maildir:LAYOUT=fs:DIRNAME=mAildir
See: https://wiki2.dovecot.org/MailLocation/Maildir
This has been working out very nicely for many years.
I'm not sure though that there is a way to 'move' to that config without having to basically migrate all your mailboxes.
MJ
On 10/06/2017 04:08 AM, David.M.Clark wrote:
Hi All, please be kind, this is my first e-mail to the list :-)
I actively support CentOS based e-mail servers running Dovecot, Sendmail, Spammassassin and 3 x SOGo based setups.
Dovecot is my goto IMAP server and have used it and modifications to it to net excellent results for years.
Then we have people who insist on only using Outlook (and in some instances the MS Live Messenger thingy).
Some shenanigans in recent years have arose with using sub-folders in the Outlook clients (2013 and 2016).
Traditionally, placing a "/" after the name of a newly desired e-mail folder has netted the result of something like:
"Rentals/"
creating:
/u/home/someuser/mail/Rentals/
Under which users then create actual e-mail folders under the "Rentals" Linux directory as such.
With Outlook 2013 and 2016 this seems to have stopped working and so I implemented a work-around where the user creates a normal folder, example "Rentals++", and I have written a cron script that trawls the $HOMEs each minute and if it finds a folder with a "++" at the end, it creates the folder as a directory, so:
/u/home/someuser/mail/Rentals++
becomes:
/u/home/someuser/mail/Rentals/
and adds this new subscription to their .subscription folder. It also sends an e-mail to the user advising that the new folder is created and they can proceed to use the "Rentals" folder for adding sub-folders (as in real text based mail folders).
The script was a quick work-around one weekend in a mad flurry to get things working and to date works but is not 'user-proof'. So I am now looking at developing an internal web interface to do the same thing and hoping with more controls, has no or far less margin for user error. Long times of Outlook folder refreshes don't help and users sometimes end up with issues that require my Linux command line help.
I have been trawling e-mail forums for some time now and have not seen any other work-arounds (or perhaps I am living under a rock) but before I embark on this web interface adventure, I just wanted to make sure I had not missed some fundamental 'bit' that I should be observing.
All servers are either CentOS 6.9 (or slightly less) and CentOS 7 with the latest updates and for things like Thunderbird and Roundcube and SOGo, work well. I need to experiment with the whole "/" for these but I am currently driven by the enforced Outlook chains.
Any input from you guys on whether this is my best approach or 'hey mate, just do this', would be much appreciated.
I am happy to share my travels script/web-wise if this is the only option to date.
--
As always, I remain at your service.
Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant)
Davrom Consulting Pty Ltd Mobile: 0418763124 PO Box 1644, Sunnybank Hills, 4109 E-mail (Work): david@davrom.com ABN: 81 096 990 804 E-mail (Priv): dmc1961@dmc1961.id.au Website: http://davrom.com Skype: dmc1961 Podcast: http://ldup.com.au Google: dmc1961@gmail.com
Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies
Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation.
Hi All,
Thanks to MJ for the suggestion as my only hope, and I have tested this with Outlook 2016 and Thunderbird today - seamless creation of folders.
Now the mammoth task of converting customers with masses of mbox folders to MailDir format using the on-line perl script: mb2md-3.20.pl
Works nicely and will be what gets us out of trouble. My only sadness is that Outlook then has to be 're-subscribed' to the folders again.... which makes sense or maybe I didn't give it long enough - its IMAP folder syncs are extremely slow compared to Thunderbird which I also tested in parallel. Thunderbird didn't need any re-sync, it just found the new subscription setup and was ready to go. But in all fairness, Outlook is still really only an Exchange compatible product for the most part.
Would be great to run both mbox and MailDir at the same time so we can do the slow user-by-user process rather than having them all down at some point completely - might look at running Dovecot on a different port maybe with a different ".conf" but thinking this may not work anyway or cause issues.... back to my server tests to see :-)
I use additional ports to 143 for external IMAP restrictions anyway so might be able to do this method somehow.
Anyway, for those interested in where I got to in my testing so far...
- Download perl script "mb2md-3.20.pl" from:
http://batleth.sapienti-sat.org/projects/mb2md
- Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to MailDir:
#mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_location = maildir:~/Maildir
Restarted Dovecot
su as user and in their home (/u/home/user on my setup):
/u/packages/mb2md-3.20.pl -m
then:
mv mail mail_old
then:
/u/packages/mb2md-3.20.pl -s mail_old -R
then:
Login with Outlook and subscribe to IMAP folders again.
Not a big fan of gazillions of little eml based text mail files in directories but hey, if it fixes our issues, maybe I need to stop wambulancing about it all and do as I am told :-)
Hope this helps - will keep this e-mail updated on findings if anything else comes to light.
On 07/10/17 00:46, David.M.Clark wrote:
Thanks for the reply MJ.
Good to hear it works in MailDir format - might need to look at this.
To date I have only had this issue with Outlook and for me personally on Fedora with Thunderbird, I tend to keep a heap of e-mail folders without sub-foldering and have my own little Linux 'utopia' of annually archiving folders etc using the Linux scripting environment, so I never have these issues..... but is a client requirement so will see what I can do from here - they were a bit excited about a web interface to do the "/" bit to get a "master/sub-folder" working so.... I still may implement this if MailDir is too much of a hassle.
Thanks again for your reply.
On 06/10/17 16:42, mj wrote:
Hi,
What we do, on the dovecot side, is:
mail_location = maildir:~/Maildir:LAYOUT=fs:DIRNAME=mAildir
See: https://wiki2.dovecot.org/MailLocation/Maildir
This has been working out very nicely for many years.
I'm not sure though that there is a way to 'move' to that config without having to basically migrate all your mailboxes.
MJ
On 10/06/2017 04:08 AM, David.M.Clark wrote:
Hi All, please be kind, this is my first e-mail to the list :-)
I actively support CentOS based e-mail servers running Dovecot, Sendmail, Spammassassin and 3 x SOGo based setups.
Dovecot is my goto IMAP server and have used it and modifications to it to net excellent results for years.
Then we have people who insist on only using Outlook (and in some instances the MS Live Messenger thingy).
Some shenanigans in recent years have arose with using sub-folders in the Outlook clients (2013 and 2016).
Traditionally, placing a "/" after the name of a newly desired e-mail folder has netted the result of something like:
"Rentals/"
creating:
/u/home/someuser/mail/Rentals/
Under which users then create actual e-mail folders under the "Rentals" Linux directory as such.
With Outlook 2013 and 2016 this seems to have stopped working and so I implemented a work-around where the user creates a normal folder, example "Rentals++", and I have written a cron script that trawls the $HOMEs each minute and if it finds a folder with a "++" at the end, it creates the folder as a directory, so:
/u/home/someuser/mail/Rentals++
becomes:
/u/home/someuser/mail/Rentals/
and adds this new subscription to their .subscription folder. It also sends an e-mail to the user advising that the new folder is created and they can proceed to use the "Rentals" folder for adding sub-folders (as in real text based mail folders).
The script was a quick work-around one weekend in a mad flurry to get things working and to date works but is not 'user-proof'. So I am now looking at developing an internal web interface to do the same thing and hoping with more controls, has no or far less margin for user error. Long times of Outlook folder refreshes don't help and users sometimes end up with issues that require my Linux command line help.
I have been trawling e-mail forums for some time now and have not seen any other work-arounds (or perhaps I am living under a rock) but before I embark on this web interface adventure, I just wanted to make sure I had not missed some fundamental 'bit' that I should be observing.
All servers are either CentOS 6.9 (or slightly less) and CentOS 7 with the latest updates and for things like Thunderbird and Roundcube and SOGo, work well. I need to experiment with the whole "/" for these but I am currently driven by the enforced Outlook chains.
Any input from you guys on whether this is my best approach or 'hey mate, just do this', would be much appreciated.
I am happy to share my travels script/web-wise if this is the only option to date.
--
As always, I remain at your service.
Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant)
Davrom Consulting Pty Ltd Mobile: 0418763124 PO Box 1644, Sunnybank Hills, 4109 E-mail (Work): david@davrom.com ABN: 81 096 990 804 E-mail (Priv): dmc1961@dmc1961.id.au Website: http://davrom.com Skype: dmc1961 Podcast: http://ldup.com.au Google: dmc1961@gmail.com
Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies
Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation.
Hi David,
I somehow remember that there is a way to get the mail_location from per-user-settings. I had this once when migrating from maildir to mdbox.
You might want to use Google and the mailing list archives for the details but in summary it went like this:
- set Dovecot to grab the mail_location from user-settings (I think we used MySQL db field - or check docs for auto:)
- have login script which converts maildir from setup A to B and update the user setting
- kick the user once again to apply the setting (I’m not sure about this)
This then automates the conversion process to happen on next user login.
Be warned though that today with all the idle phones, probably many users will re-login after you restart the server. Perhaps the scripting could handle a timer which randomizes this a little.
Philon
Am 07.10.2017 um 02:22 schrieb David.M.Clark david@davrom.com:
Hi All,
Thanks to MJ for the suggestion as my only hope, and I have tested this with Outlook 2016 and Thunderbird today - seamless creation of folders.
Now the mammoth task of converting customers with masses of mbox folders to MailDir format using the on-line perl script: mb2md-3.20.pl
Works nicely and will be what gets us out of trouble. My only sadness is that Outlook then has to be 're-subscribed' to the folders again.... which makes sense or maybe I didn't give it long enough - its IMAP folder syncs are extremely slow compared to Thunderbird which I also tested in parallel. Thunderbird didn't need any re-sync, it just found the new subscription setup and was ready to go. But in all fairness, Outlook is still really only an Exchange compatible product for the most part.
Would be great to run both mbox and MailDir at the same time so we can do the slow user-by-user process rather than having them all down at some point completely - might look at running Dovecot on a different port maybe with a different ".conf" but thinking this may not work anyway or cause issues.... back to my server tests to see :-)
I use additional ports to 143 for external IMAP restrictions anyway so might be able to do this method somehow.
Anyway, for those interested in where I got to in my testing so far...
- Download perl script "mb2md-3.20.pl" from:
http://batleth.sapienti-sat.org/projects/mb2md
- Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to MailDir:
#mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_location = maildir:~/Maildir
Restarted Dovecot
su as user and in their home (/u/home/user on my setup):
/u/packages/mb2md-3.20.pl -m
then:
mv mail mail_old
then:
/u/packages/mb2md-3.20.pl -s mail_old -R
then:
Login with Outlook and subscribe to IMAP folders again.
Not a big fan of gazillions of little eml based text mail files in directories but hey, if it fixes our issues, maybe I need to stop wambulancing about it all and do as I am told :-)
Hope this helps - will keep this e-mail updated on findings if anything else comes to light.
You can give mail location and home directory from userdb or passdb.
For userdb, return mail=something:~/something and for home use home=/path/to/home
For passdb, you can return userdb_mail and userdb_home
Aki
On October 8, 2017 at 1:14 PM Philon bytesplit@gmail.com wrote:
Hi David,
I somehow remember that there is a way to get the mail_location from per-user-settings. I had this once when migrating from maildir to mdbox.
You might want to use Google and the mailing list archives for the details but in summary it went like this:
- set Dovecot to grab the mail_location from user-settings (I think we used MySQL db field - or check docs for auto:)
- have login script which converts maildir from setup A to B and update the user setting
- kick the user once again to apply the setting (I’m not sure about this)
This then automates the conversion process to happen on next user login.
Be warned though that today with all the idle phones, probably many users will re-login after you restart the server. Perhaps the scripting could handle a timer which randomizes this a little.
Philon
Am 07.10.2017 um 02:22 schrieb David.M.Clark david@davrom.com:
Hi All,
Thanks to MJ for the suggestion as my only hope, and I have tested this with Outlook 2016 and Thunderbird today - seamless creation of folders.
Now the mammoth task of converting customers with masses of mbox folders to MailDir format using the on-line perl script: mb2md-3.20.pl
Works nicely and will be what gets us out of trouble. My only sadness is that Outlook then has to be 're-subscribed' to the folders again.... which makes sense or maybe I didn't give it long enough - its IMAP folder syncs are extremely slow compared to Thunderbird which I also tested in parallel. Thunderbird didn't need any re-sync, it just found the new subscription setup and was ready to go. But in all fairness, Outlook is still really only an Exchange compatible product for the most part.
Would be great to run both mbox and MailDir at the same time so we can do the slow user-by-user process rather than having them all down at some point completely - might look at running Dovecot on a different port maybe with a different ".conf" but thinking this may not work anyway or cause issues.... back to my server tests to see :-)
I use additional ports to 143 for external IMAP restrictions anyway so might be able to do this method somehow.
Anyway, for those interested in where I got to in my testing so far...
- Download perl script "mb2md-3.20.pl" from:
http://batleth.sapienti-sat.org/projects/mb2md
- Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to MailDir:
#mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_location = maildir:~/Maildir
Restarted Dovecot
su as user and in their home (/u/home/user on my setup):
/u/packages/mb2md-3.20.pl -m
then:
mv mail mail_old
then:
/u/packages/mb2md-3.20.pl -s mail_old -R
then:
Login with Outlook and subscribe to IMAP folders again.
Not a big fan of gazillions of little eml based text mail files in directories but hey, if it fixes our issues, maybe I need to stop wambulancing about it all and do as I am told :-)
Hope this helps - will keep this e-mail updated on findings if anything else comes to light.
Wow excellent feedback Joseph, Philon and Aki,
You have all pointed me in the right direction. Tonight I have tested Dovecot with just the minor configs I do for Dovecot setup and I have commented out the mbox/Maildir lines in 10-mail.conf from my testing:
#mail_location = #mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u #mail_location = maildir:~/Maildir
I had run up some test files (folders) under mbox using "/u/home/user/mail_test" and then renamed this to 'mail' (so /u/home/user/mail). Ran some tests with sends/receives, created folders - all good. Created mbox text folders.
Then had my initial test 'Maildir_test' populated with test folders which I renamed to /u/home/user/Maildir - and each time I restarted Outlook it was obeying the folder 'format' based on whether I had 'mail' or 'Maildir' in place.
I did see in the posts and docs on-line somewhere that someone mentioned the later implementations of Dovecot, without hard-setting either mbox or Maildir in the 10-mail.conf file, Dovecot, based on the presence of the particular directory in the user's $HOME directory obeys the rules based on mbox/Maildir respectively.
I haven't needed to touch the namespace rules as per:
https://wiki2.dovecot.org/Namespaces
so the only files that bear any changes from the installed config in conf.d are:
-rw-r--r-- 1 root root 14531 Oct 8 23:28 10-mail.conf -rw-r--r-- 1 root root 2965 Oct 7 08:49 10-master.conf -rw-r--r-- 1 root root 1728 Oct 7 08:52 10-ssl.conf
This will make migration so much easier with converting one user at a time and then getting them to start Outlook and re-sync things again.
This will save a massive job from being just that and can be done in batches of users to suit time constraints.
I will post here once again to let you all know how I fare but just proves to me over and over again the power of the Linux e-mail server with Dovecot for IMAP and being able to keep with the older SendMail MTA (been working with Sendmail since 1996 so....).
Thank you all so much for taking time to respond.
On 08/10/17 20:32, Aki Tuomi wrote:
You can give mail location and home directory from userdb or passdb.
For userdb, return mail=something:~/something and for home use home=/path/to/home
For passdb, you can return userdb_mail and userdb_home
Aki
On October 8, 2017 at 1:14 PM Philon bytesplit@gmail.com wrote:
Hi David,
I somehow remember that there is a way to get the mail_location from per-user-settings. I had this once when migrating from maildir to mdbox.
You might want to use Google and the mailing list archives for the details but in summary it went like this:
- set Dovecot to grab the mail_location from user-settings (I think we used MySQL db field - or check docs for auto:)
- have login script which converts maildir from setup A to B and update the user setting
- kick the user once again to apply the setting (I’m not sure about this)
This then automates the conversion process to happen on next user login.
Be warned though that today with all the idle phones, probably many users will re-login after you restart the server. Perhaps the scripting could handle a timer which randomizes this a little.
Philon
Am 07.10.2017 um 02:22 schrieb David.M.Clark david@davrom.com:
Hi All,
Thanks to MJ for the suggestion as my only hope, and I have tested this with Outlook 2016 and Thunderbird today - seamless creation of folders.
Now the mammoth task of converting customers with masses of mbox folders to MailDir format using the on-line perl script: mb2md-3.20.pl
Works nicely and will be what gets us out of trouble. My only sadness is that Outlook then has to be 're-subscribed' to the folders again.... which makes sense or maybe I didn't give it long enough - its IMAP folder syncs are extremely slow compared to Thunderbird which I also tested in parallel. Thunderbird didn't need any re-sync, it just found the new subscription setup and was ready to go. But in all fairness, Outlook is still really only an Exchange compatible product for the most part.
Would be great to run both mbox and MailDir at the same time so we can do the slow user-by-user process rather than having them all down at some point completely - might look at running Dovecot on a different port maybe with a different ".conf" but thinking this may not work anyway or cause issues.... back to my server tests to see :-)
I use additional ports to 143 for external IMAP restrictions anyway so might be able to do this method somehow.
Anyway, for those interested in where I got to in my testing so far...
- Download perl script "mb2md-3.20.pl" from:
http://batleth.sapienti-sat.org/projects/mb2md
- Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to MailDir:
#mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_location = maildir:~/Maildir
Restarted Dovecot
su as user and in their home (/u/home/user on my setup):
/u/packages/mb2md-3.20.pl -m
then:
mv mail mail_old
then:
/u/packages/mb2md-3.20.pl -s mail_old -R
then:
Login with Outlook and subscribe to IMAP folders again.
Not a big fan of gazillions of little eml based text mail files in directories but hey, if it fixes our issues, maybe I need to stop wambulancing about it all and do as I am told :-)
Hope this helps - will keep this e-mail updated on findings if anything else comes to light.
--
As always, I remain at your service.
Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant)
Davrom Consulting Pty Ltd Mobile: 0418763124 PO Box 1644, Sunnybank Hills, 4109 E-mail (Work): david@davrom.com ABN: 81 096 990 804 E-mail (Priv): dmc1961@dmc1961.id.au Website: http://davrom.com Skype: dmc1961 Podcast: http://ldup.com.au Google: dmc1961@gmail.com
Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies
Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation.
Hi All,
Will start using an "update" tag in my subject just to not cause confusion (mainly for yours truly).
Pursuant to my victory in finding Maildir and mbox being used if either exist (and if both still seems to favour Maildir - a mistake my end: "rmdir -f mail" fixed it), I have found one issue with not declaring the mailbox location: it won't import your new e-mail from your Inbox (/var/spool/mail/user in my case). If I run the conversion script mb2md with a '-m', I get duplicate entries in Outlook's Inbox - I understand why, just a shame it doesn't sync with what 'isn't' already there.
So yesterday and today I have done additional testing as the customer uses RoundCube and mobile devices - so attacking the same Linux server I have Thunderbird and a couple of Android e-mail clients. Outlook is the only stick in the mud with needing to manually subscribe to folders when I create them on other devices - again could be me as I don't use Outlook. (Thunderbird and other e-mail clients, all good with immediate auto-sync).
I looked at the wiki to try and run both mail_location settings with the wiki suggested:
##namespace { ## separator = / ## prefix = "#mbox/" ## location = mbox:~/mail:INBOX=/var/mail/%u ## inbox = yes ## hidden = yes ## list = no ##} ##namespace { ## separator = / ## prefix = "Maildir/" ## location = maildir:~/Maildir ## list = yes ##}
in 10-mail.conf but as you can see, I have commented it out now as it introduced some more layers of errors which I have to get back to - "Duplicate namespace prefix" was one issue.
The goal is to run the server with the existing mbox users blissfully unaware they are being converted one-by-one so it looks like I 'can' run both formats but just need to get the configs right. Was hoping the mb2md would just sync new stuff so I could cron script the Inbox check for the users that are converted to Maildir until I can throw the switch and be done with the cut-over..... or I could be dangerous and remove and import each time.... but that could lead to more dramas at the user end.
Looks like more R&D is needed so will persevere - annoying to get so close though, might see if I can just sync in the Inbox items that are 'new' - that means I am ready to roll on the customer site......
On 09/10/17 00:12, David.M.Clark wrote:
Wow excellent feedback Joseph, Philon and Aki,
You have all pointed me in the right direction. Tonight I have tested Dovecot with just the minor configs I do for Dovecot setup and I have commented out the mbox/Maildir lines in 10-mail.conf from my testing:
#mail_location = #mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u #mail_location = maildir:~/Maildir
I had run up some test files (folders) under mbox using "/u/home/user/mail_test" and then renamed this to 'mail' (so /u/home/user/mail). Ran some tests with sends/receives, created folders - all good. Created mbox text folders.
Then had my initial test 'Maildir_test' populated with test folders which I renamed to /u/home/user/Maildir - and each time I restarted Outlook it was obeying the folder 'format' based on whether I had 'mail' or 'Maildir' in place.
I did see in the posts and docs on-line somewhere that someone mentioned the later implementations of Dovecot, without hard-setting either mbox or Maildir in the 10-mail.conf file, Dovecot, based on the presence of the particular directory in the user's $HOME directory obeys the rules based on mbox/Maildir respectively.
I haven't needed to touch the namespace rules as per:
https://wiki2.dovecot.org/Namespaces
so the only files that bear any changes from the installed config in conf.d are:
-rw-r--r-- 1 root root 14531 Oct 8 23:28 10-mail.conf -rw-r--r-- 1 root root 2965 Oct 7 08:49 10-master.conf -rw-r--r-- 1 root root 1728 Oct 7 08:52 10-ssl.conf
This will make migration so much easier with converting one user at a time and then getting them to start Outlook and re-sync things again.
This will save a massive job from being just that and can be done in batches of users to suit time constraints.
I will post here once again to let you all know how I fare but just proves to me over and over again the power of the Linux e-mail server with Dovecot for IMAP and being able to keep with the older SendMail MTA (been working with Sendmail since 1996 so....).
Thank you all so much for taking time to respond.
On 08/10/17 20:32, Aki Tuomi wrote:
You can give mail location and home directory from userdb or passdb.
For userdb, return mail=something:~/something and for home use home=/path/to/home
For passdb, you can return userdb_mail and userdb_home
Aki
On October 8, 2017 at 1:14 PM Philon bytesplit@gmail.com wrote:
Hi David,
I somehow remember that there is a way to get the mail_location from per-user-settings. I had this once when migrating from maildir to mdbox.
You might want to use Google and the mailing list archives for the details but in summary it went like this:
- set Dovecot to grab the mail_location from user-settings (I think we used MySQL db field - or check docs for auto:)
- have login script which converts maildir from setup A to B and update the user setting
- kick the user once again to apply the setting (I’m not sure about this)
This then automates the conversion process to happen on next user login.
Be warned though that today with all the idle phones, probably many users will re-login after you restart the server. Perhaps the scripting could handle a timer which randomizes this a little.
Philon
Am 07.10.2017 um 02:22 schrieb David.M.Clark david@davrom.com:
Hi All,
Thanks to MJ for the suggestion as my only hope, and I have tested this with Outlook 2016 and Thunderbird today - seamless creation of folders.
Now the mammoth task of converting customers with masses of mbox folders to MailDir format using the on-line perl script: mb2md-3.20.pl
Works nicely and will be what gets us out of trouble. My only sadness is that Outlook then has to be 're-subscribed' to the folders again.... which makes sense or maybe I didn't give it long enough - its IMAP folder syncs are extremely slow compared to Thunderbird which I also tested in parallel. Thunderbird didn't need any re-sync, it just found the new subscription setup and was ready to go. But in all fairness, Outlook is still really only an Exchange compatible product for the most part.
Would be great to run both mbox and MailDir at the same time so we can do the slow user-by-user process rather than having them all down at some point completely - might look at running Dovecot on a different port maybe with a different ".conf" but thinking this may not work anyway or cause issues.... back to my server tests to see :-)
I use additional ports to 143 for external IMAP restrictions anyway so might be able to do this method somehow.
Anyway, for those interested in where I got to in my testing so far...
- Download perl script "mb2md-3.20.pl" from:
http://batleth.sapienti-sat.org/projects/mb2md
- Changed Dovecot /etc/dovecot/conf.d/10-mail.conf mbox entry to MailDir:
#mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u mail_location = maildir:~/Maildir
Restarted Dovecot
su as user and in their home (/u/home/user on my setup):
/u/packages/mb2md-3.20.pl -m
then:
mv mail mail_old
then:
/u/packages/mb2md-3.20.pl -s mail_old -R
then:
Login with Outlook and subscribe to IMAP folders again.
Not a big fan of gazillions of little eml based text mail files in directories but hey, if it fixes our issues, maybe I need to stop wambulancing about it all and do as I am told :-)
Hope this helps - will keep this e-mail updated on findings if anything else comes to light.
--
As always, I remain at your service.
Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant)
Davrom Consulting Pty Ltd Mobile: 0418763124 PO Box 1644, Sunnybank Hills, 4109 E-mail (Work): david@davrom.com ABN: 81 096 990 804 E-mail (Priv): dmc1961@dmc1961.id.au Website: http://davrom.com Skype: dmc1961 Podcast: http://ldup.com.au Google: dmc1961@gmail.com
Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies
Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation.
participants (4)
-
Aki Tuomi
-
David.M.Clark
-
mj
-
Philon