[Dovecot] Virtual Servers
Maybe a little off-topic - but I hope not too much.
Looking for some insight on setting up Dovecot under a virtual server.
In particular, I use VirtualBox - and at the moment, Ubuntu Linux.
Initial questions on configuration:
Caching. It seems to me - and I'm probably wrong - that running a Linux
in a VM on a Linux host, there would be a duplication of caching. That
is, the host server has a file cache - and the VM, which is otherwise a
standard Linux installation, is also going to try to cache its files.
This strikes me as a duplication of effort and waste of RAM. Is this
something I should devote any time to thinking about and trying to
minimize? If so, how?
Mail storage. My current mail store is a RAID-10, using the mdbox format. I wish to continue storing the mail on "raw" disks - not place the mail inside a virtual disk. Accordingly, the VM needs to reach the mail outside the VM environment - which according to conventional wisdom means NFS. My initial testing shows NFS results in a dramatically reduced performance for Dovecot. Given that this NFS access is going to be exclusively for Dovecot, and I'm only running a single server, are there any NFS or Dovecot tweaks I should implement? Is there an alternative connectivity for the VirtualBox environment I should explore?
-- Daniel
Hi Daniel,
On 06/27/2011 02:40 PM, Daniel L. Miller wrote:
Maybe a little off-topic - but I hope not too much.
Looking for some insight on setting up Dovecot under a virtual server. In particular, I use VirtualBox - and at the moment, Ubuntu Linux.
Initial questions on configuration:
Caching. It seems to me - and I'm probably wrong - that running a Linux in a VM on a Linux host, there would be a duplication of caching. That is, the host server has a file cache - and the VM, which is otherwise a standard Linux installation, is also going to try to cache its files. This strikes me as a duplication of effort and waste of RAM. Is this something I should devote any time to thinking about and trying to minimize? If so, how?
In the storage configuration of your VM, where you select the type of interface to emulate, there's a checkbox for using the Host's I/O cache.
Mail storage. My current mail store is a RAID-10, using the mdbox format. I wish to continue storing the mail on "raw" disks - not place the mail inside a virtual disk. Accordingly, the VM needs to reach the mail outside the VM environment - which according to conventional wisdom means NFS. My initial testing shows NFS results in a dramatically reduced performance for Dovecot. Given that this NFS access is going to be exclusively for Dovecot, and I'm only running a single server, are there any NFS or Dovecot tweaks I should implement? Is there an alternative connectivity for the VirtualBox environment I should explore?
If you can set aside entire block devices for use in the VM, you can create a vmdk that performs a 1:1 mapping from the virtualized disk to the given block device. The block device will be partitionable inside the VM, even if it is a partition itself. If you need to, you can access those partitions from the host with the "partx" or "kpartx" utilities (with the VM shut down, of course).
The command you want is "VBoxManage internalcommands createrawvmdk"
On the other hand, if the host and the guest need simultaneous access, you will need some form of network filesystem.
HTH,
Phil
On 6/27/2011 12:48 PM, Phil Turmel wrote:
In the storage configuration of your VM, where you select the type of interface to emulate, there's a checkbox for using the Host's I/O cache.
Does VirtualBox uses that to "trick" the guest kernel into not consuming memory for caching?
If you can set aside entire block devices for use in the VM, you can create a vmdk that performs a 1:1 mapping from the virtualized disk to the given block device. The block device will be partitionable inside the VM, even if it is a partition itself. If you need to, you can access those partitions from the host with the "partx" or "kpartx" utilities (with the VM shut down, of course).
The command you want is "VBoxManage internalcommands createrawvmdk"
I've done raw access before - but in this case I'm using a single XFS partition on a RAID10 that has multiple directories for various data needs - one of which is the mail store.
-- Daniel
On 27/06/2011 19:40, Daniel L. Miller wrote:
Maybe a little off-topic - but I hope not too much.
Looking for some insight on setting up Dovecot under a virtual server. In particular, I use VirtualBox - and at the moment, Ubuntu Linux.
Although not an option you are currently using, I'm a huge fan of linux-vservers for linux on linux virtualisation. It doesn't offer "full" virtualisation, but it's very secure and extremely lightweight. If you also hardlink all the common files then you will even reduce your RSS sizes across virtual machines due to the kernel using shared static segments
Completely useless response to your question, but just a quick thumbs up on a completely different way to slice your onion...
Good luck
Ed W
On 6/27/2011 1:03 PM, Ed W wrote:
On 27/06/2011 19:40, Daniel L. Miller wrote:
Maybe a little off-topic - but I hope not too much.
Looking for some insight on setting up Dovecot under a virtual server. In particular, I use VirtualBox - and at the moment, Ubuntu Linux. Although not an option you are currently using, I'm a huge fan of linux-vservers for linux on linux virtualisation. It doesn't offer "full" virtualisation, but it's very secure and extremely lightweight. If you also hardlink all the common files then you will even reduce your RSS sizes across virtual machines due to the kernel using shared static segments
Completely useless response to your question, but just a quick thumbs up on a completely different way to slice your onion... I'm going to need to research this and experiment a bit...
Daniel
I wouldn't worry about *duplicate cache* as far as disk goes at all.
This duplicate cache is only going to benifit your vm, if the host
machine has enough left over ram. If the host machine doesn't have
enough ram, there won't be any cache to worry about. I think this also
only applies when using a file based drive, where if you use a raw
partition it doesn't get cached, cause the vfs is bypassed.
Quoting "Daniel L. Miller" <dmiller@amfes.com>:
Maybe a little off-topic - but I hope not too much.
Looking for some insight on setting up Dovecot under a virtual
server. In particular, I use VirtualBox - and at the moment, Ubuntu
Linux.Initial questions on configuration:
Caching. It seems to me - and I'm probably wrong - that running a
Linux in a VM on a Linux host, there would be a duplication of
caching. That is, the host server has a file cache - and the VM,
which is otherwise a standard Linux installation, is also going to
try to cache its files. This strikes me as a duplication of effort
and waste of RAM. Is this something I should devote any time to
thinking about and trying to minimize? If so, how?Mail storage. My current mail store is a RAID-10, using the mdbox
format. I wish to continue storing the mail on "raw" disks - not
place the mail inside a virtual disk. Accordingly, the VM needs to
reach the mail outside the VM environment - which according to
conventional wisdom means NFS. My initial testing shows NFS results
in a dramatically reduced performance for Dovecot. Given that this
NFS access is going to be exclusively for Dovecot, and I'm only
running a single server, are there any NFS or Dovecot tweaks I
should implement? Is there an alternative connectivity for the
VirtualBox environment I should explore?-- Daniel
On 6/27/2011 2:51 PM, Patrick Domack wrote:
I wouldn't worry about *duplicate cache* as far as disk goes at all.
This duplicate cache is only going to benifit your vm, if the host machine has enough left over ram. If the host machine doesn't have enough ram, there won't be any cache to worry about. I think this also only applies when using a file based drive, where if you use a raw partition it doesn't get cached, cause the vfs is bypassed.
My primary "concern" was the VM cache. My thought was since the host was caching, there was no need for the VM to try to allocate precious VM memory for cache purposes. The goal "obviously?" being to allocate as much RAM as necessary for efficient Dovecot functioning in the VM and avoid any swap-to-disk issues.
Daniel
On Mon, 2011-06-27 at 11:40 -0700, Daniel L. Miller wrote:
Maybe a little off-topic - but I hope not too much.
Looking for some insight on setting up Dovecot under a virtual server.
In particular, I use VirtualBox - and at the moment, Ubuntu Linux.
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :)
format. I wish to continue storing the mail on "raw" disks - not place the mail inside a virtual disk. Accordingly, the VM needs to reach the
At least you have very good sense not to do that
means NFS. My initial testing shows NFS results in a dramatically reduced performance for Dovecot. Given that this NFS access is going to
Hrmmm, something amiss somewhere then, I'd put it down to the VM, many people on this list use NFS and have no problems.
there any NFS or Dovecot tweaks I should implement? Is there an
Though you have not mentioned what version you run, on 1.2.x using:
mmap_disable = yes mail_nfs_storage = yes mail_nfs_index = yes
... is a must...
Also, what else runs on your server, how many VM's and what types? The only VM I use, is personal and not work related, and then its only for my personal domain secondary MX and secondary geo located (in the U.S) DNS, it's a Xen box, but it does little work since my primaries always respond, I set it up to NFS access my mail here once playing around (seeing how much a difference being 10K Km's away makes), and the only latency I got in accessing mail, was normal and as to be expected with a 160ms each way trip, to be honest, it was faster to pop my mail that way, than using my mobile directly to my main server given mobile carriers always over subscribe.
Cheers
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :)
Don't mean to start a flame war, but your statement above is just simply inaccurate. The main difference between a virtual server and a physical server is that resources are contended with other VMs in the virtual server suitation. So for example, if Dovecot required a lot of resources, and there were another 1000 idle VMs on the same physical box as the Dovecot VM, Dovecot would run fine. What is true for any service (not just Dovecot) is that you need to get the balance right depending on your server resources requirements and sharing the physical resources with other VMs in such a way that you don't starve some critical services.
So please stop with the FUD about virtualisation :)
BTW, I'm assuming a proper virtualisation solution such as Xen Paravirtualisation.
On Tue, 2011-06-28 at 02:21 +0100, Jonathan Tripathy wrote:
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :)
Don't mean to start a flame war, but your statement above is just simply inaccurate. The main difference between a virtual server and a physical
I can show different :) but I wont be baited on this list, so stand by for a private mail in about 15 minutes
On 28/06/2011 02:21, Jonathan Tripathy wrote:
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :)
Don't mean to start a flame war, but your statement above is just simply inaccurate. The main difference between a virtual server and a physical server is that resources are contended with other VMs in the virtual server suitation. So for example, if Dovecot required a lot of resources, and there were another 1000 idle VMs on the same physical box as the Dovecot VM, Dovecot would run fine. What is true for any service (not just Dovecot) is that you need to get the balance right depending on your server resources requirements and sharing the physical resources with other VMs in such a way that you don't starve some critical services.
So please stop with the FUD about virtualisation :)
BTW, I'm assuming a proper virtualisation solution such as Xen Paravirtualisation.
I should also mention that I'm refering to VMs using direct block storage such as LVM, not VMs running off image files. Running anything off an image file is indeed going to slow your system down compared to a physical server.
A single VM system using file based storage is not going to beat a physical server. A single VM system running on direct block storage is going to be equal to a physical server in most respects. I'm not even sure if VMWare support that to be honest, but Xen sure does. :)
IMO anyway..
Noel: I tried to reply to your email off list, but it bounced :(
On Tue, 2011-06-28 at 02:54 +0100, Jonathan Tripathy wrote:
I should also mention that I'm refering to VMs using direct block storage such as LVM, not VMs running off image files. Running anything off an image file is indeed going to slow your system down compared to a physical server.
A single VM system using file based storage is not going to beat a physical server. A single VM system running on direct block storage is going to be equal to a physical server in most respects. I'm not even sure if VMWare support that to be honest, but Xen sure does. :)
Not sure how they setup the one I was telling you about, that guys off with the flu, but I'll email him
IMO anyway..
Noel: I tried to reply to your email off list, but it bounced :(
Looks like milter caught you on generic DNS
Cheers
On 6/27/2011 6:21 PM, Jonathan Tripathy wrote:
BTW, I'm assuming a proper virtualisation solution such as Xen Paravirtualisation.
"Proper"? You don't consider VirtualBox as such?
In this instance, I'm using VirtualBox to run a few Windows guests - so Xen isn't an option for me. And as far as I know, QEMU/KVM cannot run concurrently with VirtualBox. I have tried getting Windows to run under QEMU-KVM - for me at least, it was a miserable time compared with VirtualBox.
Daniel
On 6/27/2011 6:06 PM, Noel Butler wrote:
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :) Just all the mailing lists I subscribe to :)
means NFS. My initial testing shows NFS results in a dramatically reduced performance for Dovecot. Given that this NFS access is going to
Hrmmm, something amiss somewhere then, I'd put it down to the VM, many people on this list use NFS and have no problems.
Well - the wiki tells me, "Both the mmap_disable and indexing to NFS will result in a notable performance hit."
Though you have not mentioned what version you run, on 1.2.x using:
mmap_disable = yes mail_nfs_storage = yes mail_nfs_index = yes Why do people insist on specifics :) ? At the moment, 2.0.13.
Something still a bit unclear - cue Timo interjection here. The parameters listed for nfs installations (mmap_disable, doctlock_use_excl, mail_nfs_storage, mail_nfs_index) - are they necessary for data integrity, and/or do they compensate for NFS latency and improve performance? My confusion stems from the unusual? condition where the mail store is NFS based - but is otherwise dedicated to the single Dovecot instance, so simultaneous writes & locking *shouldn't* be a concern.
Particularly as I'm using mdbox, local index storage seems inappropriate (as I don't want any critical data stored within a virtual image).
Daniel
On 2011-06-27 9:06 PM, Noel Butler wrote:
On 2011-06-27 9:21 PM, Jonathan Tripathy wrote:
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :)
Don't mean to start a flame war, but your statement above is just simply inaccurate.
Please don't feed the troll...
Noel: I tried to reply to your email off list, but it bounced :(
That's what you get for replying to trolls... Noel uses brain-dead anti-spam measures. I plonked him long ago...
--
Best regards,
Charles
I think we all know who the troll is here
On Tue, 2011-06-28 at 07:14 -0400, Charles Marcus wrote:
On 2011-06-27 9:06 PM, Noel Butler wrote:
On 2011-06-27 9:21 PM, Jonathan Tripathy wrote:
I hope this is a lightly used server and does not do any real level of mail traffic else you'll soon regret running in any VM :)
Don't mean to start a flame war, but your statement above is just simply inaccurate.
Please don't feed the troll...
Noel: I tried to reply to your email off list, but it bounced :(
That's what you get for replying to trolls... Noel uses brain-dead anti-spam measures. I plonked him long ago...
participants (7)
-
Charles Marcus
-
Daniel L. Miller
-
Ed W
-
Jonathan Tripathy
-
Noel Butler
-
Patrick Domack
-
Phil Turmel