[Dovecot] NFS rquota support
For v1.1, probably applies to v1.0 code pretty easily too:
http://hg.dovecot.org/dovecot/rev/078d9dde99c8
The other rquota patches had authunix_create_default() call. Is this needed/useful?
The client is now always created+destroyed for each quota lookup. I'd think it would be faster if it was created only once, but does it then break something?
If the above isn't done, clnt_create() could be replaced with clntudp_create() and have the struct sockaddr_in cached. Patches welcome. :)
Le 02.08.2007 16:12, Timo Sirainen a écrit :
For v1.1, probably applies to v1.0 code pretty easily too:
http://hg.dovecot.org/dovecot/rev/078d9dde99c8
The other rquota patches had authunix_create_default() call. Is this needed/useful?
The client is now always created+destroyed for each quota lookup. I'd think it would be faster if it was created only once, but does it then break something?
If the above isn't done, clnt_create() could be replaced with clntudp_create() and have the struct sockaddr_in cached. Patches welcome. :)
Thanks a lot Timo !
This applied and compiled well with v1.0.3.
Then in order to use it what do we have to do ? Is this correct ?
mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota plugin: quota: fs
Thanks again
-- Nico
AAARRGGH!!
On 2 Aug 2007, at 16:19, Nicolas STRANSKY wrote:
This applied and compiled well with v1.0.3.
The patches applied (with fuzz and offsets) to 1.0.3 and compiled OK
under Solaris 10 with Sun's C compiler.
However when I try and start Dovecot I get:
Edlopen(/usr/local/dovecot-1.0.3/lib/dovecot/imap/
lib10_quota_plugin.so) failed: ld.so.1: imap: fatal: relocation
error: file /usr/local/dovecot-1.0.3/lib/dovecot/imap/
lib10_quota_plugin.so: symbol xdr_getquota_args: referenced symbol
not found
Error: imap dump-capability process returned 89
Fatal: Invalid configuration in /usr/local/dovecot-1.0.3/etc/
dovecot.conf
What am I missing?
(The especially annoying thing is that xdr_getquota_args is used in
the test program included in the source code I sent out a few days
ago and that worked... but I can't spot what is different in the
linking/runtime.)
Cheers, Mike B-(
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
On 2 Aug 2007, at 16:56, Mike Brudenell wrote:
What am I missing?
(The especially annoying thing is that xdr_getquota_args is used in
the test program included in the source code I sent out a few days
ago and that worked... but I can't spot what is different in the
linking/runtime.)
Ah! The sample program I was using #include's the file rquota_xdr.c
(generated by rpgen from the rquota.x) which defines the function
xdr_getquota_args()
However there doesn't seem to be anything in the existing 1.0.3
source or the three patches that provides this function.
Is there something missing from the patches at http://hg.dovecot.org/dovecot/diff/078d9dde99c8/src/plugins/ quota/quota-fs.c ?
Nicolas Stransky: Have you actually got it running yet, or just
compiled?
If running, did you have to do anything to
overcome this
missing symbol?
Cheers, Mike B-}
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
Sorry for the flurry of messages! ...
On 2 Aug 2007, at 17:06, Mike Brudenell wrote:
However there doesn't seem to be anything in the existing 1.0.3
source or the three patches that provides this function.Is there something missing from the patches at http://hg.dovecot.org/dovecot/diff/078d9dde99c8/src/plugins/ quota/quota-fs.c ?
I've just tried #including the rquota_cdr.c file produced by running
rpcgen on the rquota.x file (as in the code I sent out a few days ago).
That compiled and runs OK, and successfully returns a storage when
queries with the GETQUOTAROOT command:
a02 getquotaroot inbox
* QUOTAROOT "inbox" "pmb1"
* QUOTA "pmb1" (STORAGE 72970240 104857600)
a02 OK Getquotaroot completed.
*BUT* the units returned are not correct: they appear to be coming
back as bytes when the RFC says that they should be in units of 1024
octets:
http://rfc.net/rfc2087.html#s3.
The fix is, I think, trivial: just update a comment and divide by
1024 in a couple of places. See attached patch, to be used after the
other three.
The mystery of the missing xdr_getquota_args() still needs sorting
properly though.
Cheers, Mike B-)
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
On Thu, 2007-08-02 at 17:34 +0100, Mike Brudenell wrote:
I've just tried #including the rquota_cdr.c file produced by running
rpcgen on the rquota.x file (as in the code I sent out a few days ago).That compiled and runs OK, and successfully returns a storage when
queries with the GETQUOTAROOT command:a02 getquotaroot inbox * QUOTAROOT "inbox" "pmb1" * QUOTA "pmb1" (STORAGE 72970240 104857600) a02 OK Getquotaroot completed.
*BUT* the units returned are not correct: they appear to be coming
back as bytes when the RFC says that they should be in units of 1024
octets:
That's a difference between v1.0 and v1.1. v1.1 divides it by 1024 later.
The mystery of the missing xdr_getquota_args() still needs sorting
properly though.
It is a bit weird why I don't get that error, because I don't see it defined anywhere. So I suppose it has to use rpcgen after all to create the .c file. Want to send a patch? :)
Le 02.08.2007 18:06, Mike Brudenell a écrit :
Nicolas Stransky: Have you actually got it running yet, or just compiled? If running, did you have to do anything to overcome this missing symbol?
Dovecot has compiled well and is running well. I don't get a missing symbol error but quota is not working with NFS, I still get this error : Aug 2 17:20:59 rouge dovecot: IMAP(xxxxxx): quotactl(Q_GETQUOTA, nfs.domain.org:/home) failed: No such file or directory
This is Debian Etch AMD64.
-- Nico Jésus a pris soin de placer un Noir dans sa Crèche pour prévenir le racisme -- et voyez !... -+- Gilbert Cesbron (1913-1979), de petites choses. -+-
On Thu, 2007-08-02 at 19:47 +0200, Nicolas STRANSKY wrote:
Le 02.08.2007 18:06, Mike Brudenell a écrit :
Nicolas Stransky: Have you actually got it running yet, or just compiled? If running, did you have to do anything to overcome this missing symbol?
Dovecot has compiled well and is running well. I don't get a missing symbol error but quota is not working with NFS, I still get this error : Aug 2 17:20:59 rouge dovecot: IMAP(xxxxxx): quotactl(Q_GETQUOTA, nfs.domain.org:/home) failed: No such file or directory
This is Debian Etch AMD64.
I think the configure failed to set HAVE_RQUOTA to config.h.
Le 02.08.2007 19:53, Timo Sirainen a écrit :
On Thu, 2007-08-02 at 19:47 +0200, Nicolas STRANSKY wrote:
Le 02.08.2007 18:06, Mike Brudenell a écrit :
Nicolas Stransky: Have you actually got it running yet, or just compiled? If running, did you have to do anything to overcome this missing symbol? Dovecot has compiled well and is running well. I don't get a missing symbol error but quota is not working with NFS, I still get this error : Aug 2 17:20:59 rouge dovecot: IMAP(xxxxxx): quotactl(Q_GETQUOTA, nfs.domain.org:/home) failed: No such file or directory
This is Debian Etch AMD64.
I think the configure failed to set HAVE_RQUOTA to config.h.
But this is what I had from ./configure: checking rpcsvc/rquota.h usability... yes checking rpcsvc/rquota.h presence... yes checking for rpcsvc/rquota.h... yes
-- Nico Il demeure donc que l'infini n'existe que dans l'imagination, et seulement sans que l'imagination connaisse l'infini. -+- Proclus, Commentaire à Euclide -+-
Greetings -
On 3 Aug 2007, at 01:00, Nicolas STRANSKY wrote:
I think the configure failed to set HAVE_RQUOTA to config.h.
But this is what I had from ./configure: checking rpcsvc/rquota.h usability... yes checking rpcsvc/rquota.h presence... yes checking for rpcsvc/rquota.h... yes
Yup, I got that initially too but didn't get the rquota code included
in the build: I checked by searching the quota-fs.o file for "RPC"...
strings quota-fs.o | grep RPC
The string should be present if the rquota code got included.
I'm not familiar with the intricacies of GNU Autoconf etc but managed
to work out that I need to do the following in the top level of the
Dovecot source tree (above the src directory):
autoconf
autoheader
automake
If you omit the autoheader step you have configure detect that rquota
is available, but HAVE_RQUOTA still doesn't get defined: grep the
config.h file to check whether this is the case.
The autoheader step takes config.h.in and builds config.h, which then
includes the HAVE_RQUOTA definition. Compiling your source then works.
Aside: Can someone confirm whether I'm running these three utilities
in the
correct order?
Once you get the code included you may find that, like I did, the
code compiles but refuses to run. I'll try and work out a patch to
the Makefile to build the xdr file that needs including (but, as I
said before, I've never used the GNU autoconf stuff before so forgive
me if I get it wrong!).
Cheers, Mike B-)
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
On Thu, Aug 02, 2007 at 05:12:07PM +0300, Timo Sirainen wrote:
The other rquota patches had authunix_create_default() call. Is this needed/useful?
Without this, we get (Solaris 8 client talking to Solaris 10 rquotad):
IMAP(xxx): quota-fs: permission denied to rquota service
After adding a call to authunix_create_default(), quota reporting works.
I've attached patches (against 1.0.3 with the rquota support applied) to add auth support and to build the necessary XDR bits with rpcgen; the latter could probably use some work, since it assumes the availability of rpcgen and that the rquota.x file is in /usr/include/rpcsvc.
-Steven Luo
On Fri, 2007-08-03 at 20:10 -0700, Steven Luo wrote:
I've attached patches (against 1.0.3 with the rquota support applied) to add auth support
Applied directly.
and to build the necessary XDR bits with rpcgen;
I did this a bit differently. http://hg.dovecot.org/dovecot/rev/abec53314897
Le 06.08.2007 18:08, Timo Sirainen a écrit :
So both http://hg.dovecot.org/dovecot/rev/078d9dde99c8 and http://hg.dovecot.org/dovecot/rev/abec53314897 are needed to have rquota support with 1.0.x or is there anything else ?
Thanks
Nico
On Tue, 2007-08-07 at 10:29 +0200, Nicolas STRANSKY wrote:
Le 06.08.2007 18:08, Timo Sirainen a écrit :
So both http://hg.dovecot.org/dovecot/rev/078d9dde99c8 and http://hg.dovecot.org/dovecot/rev/abec53314897 are needed to have rquota support with 1.0.x or is there anything else ?
Le 07.08.2007 10:36, Timo Sirainen a écrit :
On Tue, 2007-08-07 at 10:29 +0200, Nicolas STRANSKY wrote:
Le 06.08.2007 18:08, Timo Sirainen a écrit :
So both http://hg.dovecot.org/dovecot/rev/078d9dde99c8 and http://hg.dovecot.org/dovecot/rev/abec53314897 are needed to have rquota support with 1.0.x or is there anything else ?
OK, thanks for the answer, but unfortunately, these don't apply well to 1.0.3..
-- Nico
Greetings -
On 7 Aug 2007, at 11:28, Nicolas STRANSKY wrote:
OK, thanks for the answer, but unfortunately, these don't apply
well to 1.0.3..
They applied just fine to 1.0.3 here. I started off with pristine
copies of the files from the 1.0.3 distribution then applied the
patches using GNU patch in this order:
http://hg.dovecot.org/dovecot/rev/078d9dde99c8
http://hg.dovecot.org/dovecot/rev/abec53314897
http://hg.dovecot.org/dovecot/rev/0dda1f746d63
They either applied cleanly, or with a small offset/fuzz factor.
However there is a slight problem with the final patch above if you
are compiling in a separate directory tree (eg, using GNU make's
VPATH feature).
In particular the final patch for src/plugins/quota/Makefile.am needs
to be changed from
if HAVE_RQUOTA
+quota-fs.c: rquota_xdr.c
+
rquota_xdr.c: Makefile
to
if HAVE_RQUOTA
+$(srcdir)/quota-fs.c: rquota_xdr.c
+
rquota_xdr.c: Makefile
Without this change you will get a compile-time error saying that the
quota-fs.c file couldn't be found.
(Timo: do you want me to send you this as a separate patch, or are
you happy to pick it up from the above?)
Cheers, Mike B-)
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
Le 07.08.2007 12:37, Mike Brudenell a écrit :
Greetings -
On 7 Aug 2007, at 11:28, Nicolas STRANSKY wrote:
OK, thanks for the answer, but unfortunately, these don't apply well to 1.0.3..
They applied just fine to 1.0.3 here.
You are right, they applied just fine, I just couldn't use --dry-run because these are not independent changesets.
-- Nico
It's also working here.
In my RHEL4, make crashed because it didn't get any value for SED.
export SED=/bin/sed
before autoconf solved the issue, and now it up and running.
Thanks a lot!
Aaaaaaaaaaaaagur.
El Martes, 7 de Agosto de 2007 13:17, Nicolas STRANSKY escribió:
Le 07.08.2007 12:37, Mike Brudenell a écrit :
Greetings -
On 7 Aug 2007, at 11:28, Nicolas STRANSKY wrote:
OK, thanks for the answer, but unfortunately, these don't apply well to 1.0.3..
They applied just fine to 1.0.3 here.
You are right, they applied just fine, I just couldn't use --dry-run because these are not independent changesets.
-- Joseba Torre. CIDIR Bizkaia.
In fact the patch applies well, dovecot compiles well, but rquota is still not functionnal. I have this in config.log: HAVE_RQUOTA_FALSE='#' HAVE_RQUOTA_TRUE='' #define HAVE_RQUOTA
But there is no RPC string in quota-fs.o. Where am I wrong ?
-- Nico
Greetings -
On 7 Aug 2007, at 14:36, Nicolas STRANSKY wrote:
In fact the patch applies well, dovecot compiles well, but rquota is still not functionnal. I have this in config.log: HAVE_RQUOTA_FALSE='#' HAVE_RQUOTA_TRUE='' #define HAVE_RQUOTA
But there is no RPC string in quota-fs.o. Where am I wrong ?
Try a different check: search through your config.h for RQUOTA. If
all is well you should have
#define HAVE_RQUOTA
in there. If it's not then the rquota code isn't going to get
included. (Well, that's based on my empirical observations here.)
If it's not there then try the sequence below...
This is what I did to get the build to include the rquota code:
- Apply the patches.
- cd to the top level of the distribution directory tree (above "src")
- Run: autoconf
- Run: autoheader
- Run: automake
- Run: configure
- Compile
I think the following is right (forgive me if there's anything wrong)...
autoconf does various tests to see if it is possible to use the
rquota code (eg, required configuration files and the rpcgen command
are all available); it builds the "configure" script from "configure.in"
autoheader uses "configure.in" to build "config.h.in"
automake builds the various Makefile.in files from the corresponding
Makefile.am files.
configure then builds "config.h" from "config.h.in", and the various
Makefile files from the Makefile.in files
Then you are ready to compile.
When I first tried using the patches I simply did autoconf then
automake and found, like you, that the rquota code wasn't included in
the compilation. However adding the autoheader step fixed this.
Cheers, Mike B-)
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
Le 07.08.2007 18:05, Mike Brudenell a écrit :
- Apply the patches.
- cd to the top level of the distribution directory tree (above "src")
- Run: autoconf
- Run: autoheader
- Run: automake
- Run: configure
- Compile
Thanks, this seems to work at last ! But unfortunately, I get an authentication error and I just wonder why, because the "quota" command returns the quota properly, but not dovecot..
Aug 7 19:34:36 rouge dovecot: IMAP(stransky): quota-fs: remote rquota call failed: RPC: Authentication error
-- Nico
On Thu, 2007-08-09 at 12:00 +0200, Nicolas STRANSKY wrote:
Le 07.08.2007 18:05, Mike Brudenell a écrit :
- Apply the patches.
- cd to the top level of the distribution directory tree (above "src")
- Run: autoconf
- Run: autoheader
- Run: automake
- Run: configure
- Compile
Thanks, this seems to work at last ! But unfortunately, I get an authentication error and I just wonder why, because the "quota" command returns the quota properly, but not dovecot..
Aug 7 19:34:36 rouge dovecot: IMAP(stransky): quota-fs: remote rquota call failed: RPC: Authentication error
Did you include http://hg.dovecot.org/dovecot/rev/0f6d4f8dd587
Le 09.08.2007 12:45, Timo Sirainen a écrit :
Aug 7 19:34:36 rouge dovecot: IMAP(stransky): quota-fs: remote rquota call failed: RPC: Authentication error
Did you include http://hg.dovecot.org/dovecot/rev/0f6d4f8dd587
You are right, this was not included, thanks ! Now I have included this patch and it works better, I mean there is no more error and thunderbird for example says that the server is supporting quotas.
So, to sum up, the four needed patches are: http://hg.dovecot.org/dovecot/rev/078d9dde99c8 http://hg.dovecot.org/dovecot/rev/abec53314897 http://hg.dovecot.org/dovecot/rev/0dda1f746d63 http://hg.dovecot.org/dovecot/rev/0f6d4f8dd587
But now it returns no value, and always says the status is 0% used (0 out of 0Ko used)... should I configure something else ? Thunderbird also says the quota root is : ""
Thanks
Nico
Sorry to be so clueless, but all the activity about rquotad drives me to admit my puzzlement (or ignorance)... I run rquotad on my mail server that also runs DC....rquotad is used by the other 3 hosts (a login/FTP server, a mailing list server and a user mgmnt server) that NFS mount the folder and inbox filesystem...which are under filesystem quota on the mail server where they are physically resident. AFAIK it is not queried on the mail server...after all, filesystem quota is running there. How/why does DC need/use rquotad?
Timo Sirainen wrote:
On Tue, 2007-08-07 at 10:29 +0200, Nicolas STRANSKY wrote:
Le 06.08.2007 18:08, Timo Sirainen a écrit :
So both http://hg.dovecot.org/dovecot/rev/078d9dde99c8 and http://hg.dovecot.org/dovecot/rev/abec53314897 are needed to have rquota support with 1.0.x or is there anything else ?
--
Stewart Dean, Unix System Admin, Henderson Computer Resources
Center of Bard College, Annandale-on-Hudson, New York 12504
sdean@bard.edu voice: 845-758-7475, fax: 845-758-7035
Sorry to be so clueless, but all the activity about rquotad drives me to admit my puzzlement (or ignorance)... I run rquotad on my mail server that also runs DC....rquotad is used by the other 3 hosts (a login/FTP server, a mailing list server and a user mgmnt server) that NFS mount the folder and inbox filesystem...which are under filesystem quota on the mail server where they are physically resident. AFAIK it is not queried on the mail server...after all, filesystem quota is running there. How/why does DC need/use rquotad?
You can have your mail spool located on an NFS server. In that case the dovecot server needs to use rquotad to get quota info from the nfs server.
Cor
Greetings -
On 7 Aug 2007, at 13:54, Stewart Dean wrote:
Sorry to be so clueless, but all the activity about rquotad drives
me to admit my puzzlement (or ignorance)... I run rquotad on my mail server that also runs DC....rquotad is
used by the other 3 hosts (a login/FTP server, a mailing list
server and a user mgmnt server) that NFS mount the folder and inbox
filesystem...which are under filesystem quota on the mail server
where they are physically resident. AFAIK it is not queried on the
mail server...after all, filesystem quota is running there. How/ why does DC need/use rquotad?
I think you have answered your own questions actually! ...
rquotad is used to allow other machines that NFS-mount a filestore.
The rquotad daemon runs on the machine serving up the filestore to
the other clients.
You say that your mail filestore physically resides on your mail
server. That means it is a locally attached disk (not mounted using
NFS from some other server), and so Dovecot can, and does, obtain
quotas directly from it: it does not need to ask an rquotad daemon.
In contrast here we have the mailstore on a NetApp filer, and mount
it over NFS on our machines running Dovecot. In this case Dovecot
cannot query the quota directly because the filestore isn't on
locally attached disk. Instead it must use an RPC (Remote Procedure
Call) to ask the rquotad daemon running on the file server (in this
case the NetApp filer) what the quota usage and limits are.
Cheers, Mike B-)
-- The Computing Service, University of York, Heslington, York Yo10 5DD, UK Tel:+44-1904-433811 FAX:+44-1904-433740
- Unsolicited commercial e-mail is NOT welcome at this e-mail address. *
participants (7)
-
Cor Bosman
-
Joseba Torre
-
Mike Brudenell
-
Nicolas STRANSKY
-
Steven Luo
-
Stewart Dean
-
Timo Sirainen