[Dovecot] expire plugin + --exec-mail in 1.2.6 version
Hello list,
After upgrade to dovecot version 1.2.6 I have noticed some quota malfunctions(users become often overquota). Looking at logs closely I have found that quota value resets to general dovecot-config( quota = maildir:Mailbox quota quota_rule = *:storage=500M quota_rule2 = Trash:storage=10%% ), after dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh command. The problem that some users have their own quotas stored in ldap.
For example, user sr@domain.com have maildirsize before running expire-tool(1G value from ldap): # head -2 /data/mail/domain.com/sr/data/maildirsize 10485760000S 836423205 8285
Then I run expire-tool with mail_debug enabled: # /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh Info: Loading modules from directory: /usr/lib/dovecot/imap Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so Info: Quota root: name=Mailbox quota backend=maildir args= Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) messages=0 Info: Quota warning: bytes=471859200 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Info: expire: pattern=Trash type=expunge secs=2592000 Info: expire: pattern=Spam type=expunge secs=2592000 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: koshikov.n@domain.com/Spam: seq=1 uid=1869: Expunge Info: koshikov.n@domain.com/Spam: seq=2 uid=1870: Expunge Info: koshikov.n@domain.com/Spam: seq=3 uid=1871: Expunge Info: koshikov.n@domain.com/Spam: seq=4 uid=1872: Expunge Info: koshikov.n@domain.com/Spam: seq=5 uid=1873: Expunge Info: koshikov.n@domain.com/Spam: seq=6 uid=1874: Expunge Info: koshikov.n@domain.com/Spam: seq=7 uid=1875: Expunge Info: koshikov.n@domain.com/Spam: seq=8 uid=1876: Expunge Info: koshikov.n@domain.com/Spam: seq=9 uid=1877: Expunge Info: koshikov.n@domain.com/Spam: seq=10 uid=1878: Expunge Info: koshikov.n@domain.com/Spam: seq=11 uid=1879: Expunge Info: koshikov.n@domain.com/Spam: seq=12 uid=1880: Expunge Info: koshikov.n@domain.com/Spam: timestamp 1256618680 (Tue Oct 27 06:44:40 2009) -> 1256716674 (Wed Oct 28 09:57:54 2009) Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/fsb Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/fsb/data, index=, control=, inbox=/data/mail/domain.com/fsb/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/fsb/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: fsb@domain.com/Trash: seq=1 uid=578: Expunge Info: fsb@domain.com/Trash: seq=2 uid=579: Expunge Info: fsb@domain.com/Trash: seq=3 uid=580: Expunge Info: fsb@domain.com/Trash: seq=4 uid=581: Expunge Info: fsb@domain.com/Trash: seq=5 uid=582: Expunge Info: fsb@domain.com/Trash: seq=6 uid=583: Expunge Info: fsb@domain.com/Trash: seq=7 uid=584: Expunge Info: fsb@domain.com/Trash: seq=8 uid=585: Expunge Info: fsb@domain.com/Trash: seq=9 uid=586: Expunge Info: fsb@domain.com/Trash: seq=10 uid=587: Expunge Info: fsb@domain.com/Trash: seq=11 uid=588: Expunge Info: fsb@domain.com/Trash: seq=12 uid=589: Expunge Info: fsb@domain.com/Trash: seq=13 uid=590: Expunge Info: fsb@domain.com/Trash: timestamp 1256627284 (Tue Oct 27 09:08:04 2009) -> 1256732991 (Wed Oct 28 14:29:51 2009) Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/arsen Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/arsen/data, index=, control=, inbox=/data/mail/domain.com/arsen/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/arsen/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: arsen@domain.com/Trash: seq=1 uid=14: Expunge Info: arsen@domain.com/Trash: seq=2 uid=15: Expunge Info: arsen@domain.com/Trash: seq=3 uid=16: Expunge Info: arsen@domain.com/Trash: seq=4 uid=17: Expunge Info: arsen@domain.com/Trash: seq=5 uid=18: Expunge Info: arsen@domain.com/Trash: seq=6 uid=19: Expunge Info: arsen@domain.com/Trash: seq=7 uid=20: Expunge Info: arsen@domain.com/Trash: seq=8 uid=21: Expunge Info: arsen@domain.com/Trash: seq=9 uid=22: Expunge Info: arsen@domain.com/Trash: seq=10 uid=23: Expunge Info: arsen@domain.com/Trash: seq=11 uid=24: Expunge Info: arsen@domain.com/Trash: timestamp 1256630371 (Tue Oct 27 09:59:31 2009) -> 1256733711 (Wed Oct 28 14:41:51 2009) Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: koshikov.n@domain.com/Trash: seq=1 uid=2561: Expunge Info: koshikov.n@domain.com/Trash: timestamp 1256709279 (Wed Oct 28 07:54:39 2009) -> 1256795118 (Thu Oct 29 07:45:18 2009) Info: auth input: quota_rule=*:bytes=10485760000 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/sr Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/sr/data, index=, control=, inbox=/data/mail/domain.com/sr/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/sr/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: sr@domain.com/Spam: seq=1 uid=405: Expunge Info: sr@domain.com/Spam: seq=2 uid=406: Expunge Info: sr@domain.com/Spam: seq=3 uid=407: Expunge Info: sr@domain.com/Spam: timestamp 1256710764 (Wed Oct 28 08:19:24 2009) -> 1256980000 (Sat Oct 31 11:06:40 2009) Info: kolotilo.m@domain.com/Trash: stop, expire time in future: Wed Oct 28 10:27:41 2009
And after expire-tool ends it's stuff, user sr@domain.com have(512M from dovecot-config): 524288000SS 836423205 8285
As you can see - quota reset to default value. In changes to version 1.2.6 I see
- dovecot --exec-mail was broken.
Is fixes to 1.2.6 touch userdb lookups from --exec-mail ? With 1.2.4 expire-tool works correct and I have no config changes to dovecot setup.
If dovecot -n output is needed, please let me know.
Thanks.
On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote:
The problem that some users have their own quotas stored in ldap.
So the quota_rule should be returned by userdb lookup from ldap, right?
Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data
This should have listed quota_rule.
If dovecot -n output is needed, please let me know.
If the above doesn't help, then yeah, dovecot -n and dovecot-ldap.conf contents.
On Thu, 29 Oct 2009 13:10:37 -0400 Timo Sirainen tss@iki.fi wrote:
On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote:
The problem that some users have their own quotas stored in ldap.
So the quota_rule should be returned by userdb lookup from ldap, right?
Yes, and it's working fine with deliver and general imap process, it only failed(reset to general quota_rule) after expire-rool.sh script finished his duties.
Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data
This should have listed quota_rule.
The problematic user is sr, and it has quota_rule option(running with --test)
(Thu Oct 29 07:45:18 2009) Info: auth input: quota_rule=*:bytes=10485760000 ^^^^^^^^^^^^^^^^^^^^^^^^^ returned from ldap Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/sr Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/sr/data, index=, control=, inbox=/data/mail/domain.com/sr/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/sr/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: sr@domain.com/Spam: seq=1 uid=405: Expunge Info: sr@domain.com/Spam: seq=2 uid=406: Expunge Info: sr@domain.com/Spam: seq=3 uid=407: Expunge Info: sr@domain.com/Spam: timestamp 1256710764 (Wed Oct 28 08:19:24 2009) -> 1256980000 (Sat Oct 31 11:06:40 2009)
If dovecot -n output is needed, please let me know.
If the above doesn't help, then yeah, dovecot -n and dovecot-ldap.conf contents.
dovecot -n: # 1.2.6: /etc/dovecot/dovecot.conf # OS: Linux 2.6.26-gentoo-r4 i686 Gentoo Base System release 1.12.11.1 log_path: /var/log/dovecot/dovecot-error.log info_log_path: /var/log/dovecot/dovecot.log protocols: imaps managesieve ssl_cert_file: /etc/ssl/dovecot/imaps.crt ssl_key_file: /etc/ssl/dovecot/imaps.key login_dir: /var/run/dovecot/login login_executable(default): /usr/libexec/dovecot/imap-login login_executable(imap): /usr/libexec/dovecot/imap-login login_executable(managesieve): /usr/libexec/dovecot/managesieve-login login_greeting: Server ready. login_processes_count: 20 login_max_processes_count: 512 mail_max_userip_connections(default): 20 mail_max_userip_connections(imap): 20 mail_max_userip_connections(managesieve): 10 first_valid_uid: 8 last_valid_uid: 8 first_valid_gid: 12 last_valid_gid: 12 mail_drop_priv_before_exec: yes mail_executable(default): /usr/libexec/dovecot/imap mail_executable(imap): /usr/libexec/dovecot/imap mail_executable(managesieve): /usr/libexec/dovecot/managesieve mail_plugins(default): quota imap_quota trash expire zlib autocreate virtual mail_plugins(imap): quota imap_quota trash expire zlib autocreate virtual mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/imap mail_plugin_dir(imap): /usr/lib/dovecot/imap mail_plugin_dir(managesieve): /usr/lib/dovecot/managesieve imap_client_workarounds(default): delay-newmail imap_client_workarounds(imap): delay-newmail imap_client_workarounds(managesieve): namespace: type: private separator: / location: maildir:~/data inbox: yes list: yes subscriptions: yes namespace: type: private separator: / prefix: Company/ location: virtual:/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ list: yes lda: postmaster_address: postmaster@domain.com hostname: mail.domain.com mail_plugins: quota trash expire sieve virtual quota_full_tempfail: yes sendmail_path: /usr/sbin/sendmail auth_socket_path: /var/run/dovecot/auth-master log_path: /var/log/dovecot/dovecot-deliver.log info_log_path: /var/log/dovecot/dovecot-deliver.log global_script_path: /etc/dovecot/sieve/default.sieve sieve_global_dir: /etc/dovecot/sieve auth default: mechanisms: plain login default_realm: domain.com cache_size: 10240 cache_negative_ttl: 0 user: dovecot_auth master_user_separator: * worker_max_count: 50 passdb: driver: passwd-file args: /etc/dovecot/passdb/master.pwd master: yes passdb: driver: passwd-file args: /etc/dovecot/passdb/users.pwd passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: prefetch userdb: driver: ldap args: /etc/dovecot/dovecot-userdb-ldap.conf userdb: driver: passwd-file args: /etc/dovecot/passdb/users.pwd socket: type: listen client: path: /var/run/dovecot/auth-client mode: 432 user: mail group: dovecot_auth master: path: /var/run/dovecot/auth-master mode: 384 user: mail group: mail plugin: quota_warning: storage=90%% /etc/dovecot/plugins/quota_warning.sh 90 quota: maildir:Mailbox quota quota_rule: *:storage=500M quota_rule2: Trash:storage=10%% trash: /etc/dovecot/plugins/dovecot-trash.conf expire: Trash 30 Spam 30 expire_dict: proxy::expire autocreate: Drafts autocreate2: Sent autocreate3: Spam autocreate4: Trash autosubscribe: Drafts autosubscribe2: Sent autosubscribe3: Spam autosubscribe4: Trash sieve: ~/.dovecot.sieve sieve_dir: ~/sieve sieve_extensions: +notify sieve_before: /etc/dovecot/sieve/default.sieve dict: expire: sqlite:/etc/dovecot/plugins/expire.conf
dovecot-ldap.conf:
uris = ldaps://off-dc-1 ldaps://off-dc-2 dn = imap@ dnpass = qweqwe auth_bind = yes ldap_version = 3 base = OU=Company,DC=company,DC=intranet pass_attrs = =userdb_uid=8, =userdb_gid=12,=userdb_home=/data/mail/%d/%n, =userdb_mail=maildir:~/data, maxStorage=userdb_quota_rule=*:bytes=%$
pass_filter = (&(mail=%u)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))
user_attrs = =uid=8, =gid=12,=home=/data/mail/%d/%n, =mail=maildir:~/data, maxStorage=quota_rule=*:bytes=%$
user_filter = (&(mail=%u)(!(userAccountControl:1.2.840.113556.1.4.803:=2)))
On Thu, 29 Oct 2009 13:10:37 -0400 Timo Sirainen tss@iki.fi wrote:
On Wed, 2009-10-28 at 10:30 +0200, Nikita Koshikov wrote:
The problem that some users have their own quotas stored in ldap.
So the quota_rule should be returned by userdb lookup from ldap, right?
One more clean example. For user koshikov.n@domain.com I change quota in AD to 600M(default user quota is 512M,). After successful login to imap account, maildirsize is(first 2 lines):
629145600S 260658345 5484
So, quota value read from ldap database and applied to user's mailbox. Then I run /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh --test:
Info: Loading modules from directory: /usr/lib/dovecot/imap Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so Info: Quota root: name=Mailbox quota backend=maildir args= Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) messages=0 Info: Quota warning: bytes=471859200 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Info: expire: pattern=Trash type=expunge secs=2592000 Info: expire: pattern=Spam type=expunge secs=2592000 Info: auth input: quota_rule=*:bytes=629145600 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Agromat/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2 Info: koshikov.n@domain.com/Spam: seq=1 uid=1933: Expunge Info: koshikov.n@domain.com/Spam: timestamp 1256902423 (Fri Oct 30 13:33:43 2009) -> 1256903611 (Fri Oct 30 13:53:31 2009)
Quota_rule are different from auth section and quota root\rule section. After expire-tool expunges e-mails, the maildirsize file become:
524288000S 260686038 5487
Which is default quota_rule value, not from ldap database.
On Fri, 2009-10-30 at 14:03 +0200, Nikita Koshikov wrote:
The problem that some users have their own quotas stored in ldap.
See if the attached patch helps? I'm a bit afraid that it could also break something. If it does, there's really no easy way to get this fixed before v2.0.
On Fri, 30 Oct 2009 13:16:47 -0400 Timo Sirainen tss@iki.fi wrote:
On Fri, 2009-10-30 at 14:03 +0200, Nikita Koshikov wrote:
The problem that some users have their own quotas stored in ldap.
See if the attached patch helps? I'm a bit afraid that it could also break something. If it does, there's really no easy way to get this fixed before v2.0.
The patch helped with quotas, but it raises new error - "segmentation error"
Here details: # /usr/sbin/dovecot --exec-mail ext /usr/libexec/dovecot/expire-tool.sh Info: Loading modules from directory: /usr/lib/dovecot/imap Info: Module loaded: /usr/lib/dovecot/imap/lib10_quota_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib11_trash_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_autocreate_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_expire_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_virtual_plugin.so Info: Module loaded: /usr/lib/dovecot/imap/lib20_zlib_plugin.so
-->koshikov.n@domain.com - have own quota in ldap, expire-tool is working fine:
Info: expire: pattern=Trash type=expunge secs=2592000 Info: expire: pattern=Spam type=expunge secs=2592000 Info: auth input: quota_rule=*:bytes=629145600 Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/koshikov.n Info: auth input: mail=maildir:~/data Info: Quota root: name=Mailbox quota backend=maildir args= Info: Quota rule: root=Mailbox quota mailbox=* bytes=629145600 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=62914560 (10%) messages=0 Info: Quota warning: bytes=566231040 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/koshikov.n/data, index=, control=, inbox=/data/mail/domain.com/koshikov.n/data Info: Namespace: type=private, prefix=Company/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/koshikov.n/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2
--> bunkina.l@domain.com have default quota value, expire-tool is working fine:
Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/bunkina.l Info: auth input: mail=maildir:~/data Info: Quota root: name=Mailbox quota backend=maildir args= Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) messages=0 Info: Quota warning: bytes=471859200 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Info: Namespace: type=private, prefix=, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes Info: maildir: data=~/data Info: maildir++: root=/data/mail/domain.com/bunkina.l/data, index=, control=, inbox=/data/mail/domain.com/bunkina.l/data Info: Namespace: type=private, prefix=Company/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no Info: virtual: data=/var/mail/virtual:INDEX=MEMORY:LAYOUT=maildir++ Info: maildir++: root=/var/mail/virtual, index=, control=, inbox= Info: Namespace : Using permissions from /data/mail/domain.com/bunkina.l/data: mode=0700 gid=-1 Info: trash plugin: Added 'Trash' with priority 1 Info: trash plugin: Added 'Spam' with priority 2
--> podburtniy@domain.com - have default quota, this account cause segmentation error, while running expire-tool:
Info: auth input: uid=8 Info: auth input: gid=12 Info: auth input: home=/data/mail/domain.com/podburtniy Info: auth input: mail=maildir:~/data Info: Quota root: name=Mailbox quota backend=maildir args= Info: Quota rule: root=Mailbox quota mailbox=* bytes=524288000 messages=0 Info: Quota rule: root=Mailbox quota mailbox=Trash bytes=52428800 (10%) messages=0 Info: Quota warning: bytes=471859200 (90%) messages=0 command=/etc/dovecot/plugins/quota_warning.sh 90 Ошибка сегментирования
The only difference with mailbox podburtniy@domain.com that it's ldap mail field is different from samaccountname:
podburtniy@domain.com mail: podburtniy@domain.com sAMAccountName: podburtn
koshikov.n@domain.com mail: koshikov.n@domain.com sAMAccountName: koshikov.n
But, I'm using only mail/maxstorage/useraccountcontrol attributes for mail staff. Users login to their accounts using full e-mail address and AD password.
On Sat, 2009-10-31 at 10:42 +0200, Nikita Koshikov wrote:
The patch helped with quotas, but it raises new error - "segmentation error"
gdb backtrace would be helpful. But how to actually get it could be a bit difficult, since I guess kernel thinks expire-tool is a setuid program.
There are two ways I guess:
a) Apply attached diff1 and see if core dump gets written. Once you get a core, use "gdb expire-tool core", "bt full".
b) If not, apply diff2 and start expire-tool. Then look up its pid and run "gdb -p <pid>", "cont", wait for crash, "bt full".
Hi,
I now find more information about the locking problem.
Users have multiple active imap connection from same machine for example:
13946 ? S 0:00 imap [user1 192.168.1.11] 14491 ? S 0:00 imap [user2 192.168.1.11] 28504 ? S 0:00 imap [user1 192.168.1.11] 29040 ? S 0:00 imap [user2 192.168.1.11] 29041 ? S 0:00 imap [user2 192.168.1.11]
The /var/spool/mail is on NFS share.
Running strace on 13946 and 28504:
# strace -p 13946
Process 13946 attached - interrupt to quit
gettimeofday({1257443764, 885542}, {0, 0}) = 0
gettimeofday({1257443764, 885682}, NULL) = 0
epoll_wait(6, {}, 4, 1079) = 0
gettimeofday({1257443765, 965259}, {0, 0}) = 0
gettimeofday({1257443765, 965392}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443770, 965380}, {0, 0}) = 0
gettimeofday({1257443770, 965531}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443775, 965409}, {0, 0}) = 0
gettimeofday({1257443775, 965555}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443780, 965498}, {0, 0}) = 0
gettimeofday({1257443780, 965639}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443785, 965779}, {0, 0}) = 0
gettimeofday({1257443785, 965924}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443790, 969831}, {0, 0}) = 0
gettimeofday({1257443790, 969969}, NULL) = 0
epoll_wait(6, {}, 4, 3111) = 0
gettimeofday({1257443794, 89867}, {0, 0}) = 0
writev(1, [{"* OK Still here"..., 15}, {"\r\n"..., 2}], 2) = 17
gettimeofday({1257443794, 90356}, NULL) = 0
stat64("/var/mail/user1", {st_mode=S_IFREG|0600, st_size=8603552, ...}) = 0
gettimeofday({1257443794, 103769}, NULL) = 0
epoll_wait(6, {}, 4, 1866) = 0
gettimeofday({1257443795, 969925}, {0, 0}) = 0
gettimeofday({1257443795, 970063}, NULL) = 0
epoll_wait(6, {}, 4, 4999) = 0
gettimeofday({1257443800, 970025}, {0, 0}) = 0
gettimeofday({1257443800, 970156}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443805, 970088}, {0, 0}) = 0
gettimeofday({1257443805, 970221}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
gettimeofday({1257443810, 970121}, {0, 0}) = 0
gettimeofday({1257443810, 970261}, NULL) = 0
epoll_wait(6, {}, 4, 5000) = 0
# strace -p 28504 Process 28504 attached - interrupt to quit gettimeofday({1257443943, 506081}, {0, 0}) = 0 gettimeofday({1257443943, 506235}, NULL) = 0 epoll_wait(6, {}, 4, 22730) = 0 gettimeofday({1257443966, 226716}, {0, 0}) = 0 gettimeofday({1257443966, 226855}, NULL) = 0 epoll_wait(6, {}, 4, 10) = 0 gettimeofday({1257443966, 236612}, {0, 0}) = 0 stat64("/var/mail/user1", {st_mode=S_IFREG|0600, st_size=8603552, ...}) = 0 gettimeofday({1257443966, 259124}, NULL) = 0 epoll_wait(6,
Both imap connections try to access same /var/mail/user1 file. This caused very slow performance.
Why are there multiple active imap connections? How to stop it?
Thanks,
Simon
On Thu, 2009-11-05 at 10:01 -0800, Simon Gao wrote:
stat64("/var/mail/user1", {st_mode=S_IFREG|0600, st_size=8603552, ...}) = 0
These stats are fine.
Both imap connections try to access same /var/mail/user1 file. This caused very slow performance.
Well, it usually shouldn't.. Although it depends on IMAP client I guess.
Why are there multiple active imap connections? How to stop it?
Some clients do that. For example with Thunderbird you can configure how many connections it creates.
As for your other email..:
We just have another user reporting problem. There is no lock on /var/spool/mail/<user> file.
..
fcntl64(8, F_SETLKW64, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}, 0xff9f4db4) = ? ERESTARTSYS (To be restarted)
The straces show that the file is locked. Maybe you've just disabled dotlocks? Or anyway read locks don't create the dotlock file so that could be it also.
So if some processes are trying to do fcntl locks, you could find out what process actually has the file locked and what it's doing.
Anyway, the easiest solution would be to just get rid of mbox and switch to maildir. No more locking troubles.
On Thu, 05 Nov 2009 12:52:08 -0500 Timo Sirainen tss@iki.fi wrote:
On Sat, 2009-10-31 at 10:42 +0200, Nikita Koshikov wrote:
The patch helped with quotas, but it raises new error - "segmentation error"
gdb backtrace would be helpful. But how to actually get it could be a bit difficult, since I guess kernel thinks expire-tool is a setuid program.
There are two ways I guess:
a) Apply attached diff1 and see if core dump gets written. Once you get a core, use "gdb expire-tool core", "bt full".
This didn't create core file.
b) If not, apply diff2 and start expire-tool. Then look up its pid and run "gdb -p <pid>", "cont", wait for crash, "bt full".
I don't understand why gdb says "no debugging symbols found" for some dovecot's executables, all binaries under /usr/libexec/dovecot have "not stripped"
GNU gdb 6.8 Copyright (C) 2008 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "i686-pc-linux-gnu". Attaching to process 26407 Reading symbols from /usr/libexec/dovecot/expire-tool...(no debugging symbols found)...done. Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/libdl.so.2 Reading symbols from /lib/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib/librt.so.1 Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done. Loaded symbols for /lib/ld-linux.so.2 Reading symbols from /lib/libpthread.so.0... (no debugging symbols found)...done. [Thread debugging using libthread_db enabled] [New Thread 0xb7f02ac0 (LWP 26407)] Loaded symbols for /lib/libpthread.so.0 (no debugging symbols found) 0xffffe424 in __kernel_vsyscall () (gdb) cont Continuing. (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found) (no debugging symbols found)
Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0xb7f02ac0 (LWP 26407)] 0xb7f88749 in ?? () from /lib/libc.so.6 (gdb) bt full #0 0xb7f88749 in ?? () from /lib/libc.so.6 No symbol table info available. Cannot access memory at address 0xbf690ff8 (gdb) quit The program is running. Quit anyway (and detach it)? (y or n) y Detaching from program: /usr/libexec/dovecot/expire-tool, process 26407
On Fri, 2009-11-06 at 10:27 +0200, Nikita Koshikov wrote:
I don't understand why gdb says "no debugging symbols found" for some dovecot's executables, all binaries under /usr/libexec/dovecot have "not stripped"
..
(gdb) bt full #0 0xb7f88749 in ?? () from /lib/libc.so.6 No symbol table info available. Cannot access memory at address 0xbf690ff8
Yeah, something in your system has broken gdb.. Makes the debugging annoyingly difficult.
On Fri, 06 Nov 2009 18:42:49 -0500 Timo Sirainen tss@iki.fi wrote:
On Fri, 2009-11-06 at 10:27 +0200, Nikita Koshikov wrote:
I don't understand why gdb says "no debugging symbols found" for some dovecot's executables, all binaries under /usr/libexec/dovecot have "not stripped"
..
(gdb) bt full #0 0xb7f88749 in ?? () from /lib/libc.so.6 No symbol table info available. Cannot access memory at address 0xbf690ff8
Yeah, something in your system has broken gdb.. Makes the debugging annoyingly difficult.
I have wrote simple wrapper that removes maildirsize if the messages were expunged from user's mailbox and if user has non-default quota value. Is there any more convenient way to force dovecot recalculate quota size ? I have overview IMAP-QOUTA rfc, but seemed that it lacks such imap command, maybe some internal dovecot command exist?
On Mon, 2009-11-23 at 18:01 +0200, Nikita Koshikov wrote:
On Fri, 06 Nov 2009 18:42:49 -0500 Timo Sirainen tss@iki.fi wrote:
On Fri, 2009-11-06 at 10:27 +0200, Nikita Koshikov wrote:
I don't understand why gdb says "no debugging symbols found" for some dovecot's executables, all binaries under /usr/libexec/dovecot have "not stripped"
..
(gdb) bt full #0 0xb7f88749 in ?? () from /lib/libc.so.6 No symbol table info available. Cannot access memory at address 0xbf690ff8
Yeah, something in your system has broken gdb.. Makes the debugging annoyingly difficult.
You managed to get backtraces from the other bug. What did you do differently? Maybe you can get backtrace from this bug now too?
I have wrote simple wrapper that removes maildirsize if the messages were expunged from user's mailbox and if user has non-default quota value. Is there any more convenient way to force dovecot recalculate quota size ? I have overview IMAP-QOUTA rfc, but seemed that it lacks such imap command, maybe some internal dovecot command exist?
No, there's no other way.
On Mon, 23 Nov 2009 12:01:53 -0500 Timo Sirainen tss@iki.fi wrote:
On Mon, 2009-11-23 at 18:01 +0200, Nikita Koshikov wrote:
On Fri, 06 Nov 2009 18:42:49 -0500 Timo Sirainen tss@iki.fi wrote:
On Fri, 2009-11-06 at 10:27 +0200, Nikita Koshikov wrote:
I don't understand why gdb says "no debugging symbols found" for some dovecot's executables, all binaries under /usr/libexec/dovecot have "not stripped"
..
(gdb) bt full #0 0xb7f88749 in ?? () from /lib/libc.so.6 No symbol table info available. Cannot access memory at address 0xbf690ff8
Yeah, something in your system has broken gdb.. Makes the debugging annoyingly difficult.
You managed to get backtraces from the other bug. What did you do differently? Maybe you can get backtrace from this bug now too?
I think gdb can't access memory due to gentoo kernel patches, so attaching to process is not working. Also, I can't change kernel on production server for testing purposes. As I wrote above, core files not present in users's homedir, even with patch you provided.
I have wrote simple wrapper that removes maildirsize if the messages were expunged from user's mailbox and if user has non-default quota value. Is there any more convenient way to force dovecot recalculate quota size ? I have overview IMAP-QOUTA rfc, but seemed that it lacks such imap command, maybe some internal dovecot command exist?
No, there's no other way.
Clear, thanks.
On Fri, 30 Oct 2009 13:16:47 -0400 Timo Sirainen tss@iki.fi wrote:
On Fri, 2009-10-30 at 14:03 +0200, Nikita Koshikov wrote:
The problem that some users have their own quotas stored in ldap.
See if the attached patch helps? I'm a bit afraid that it could also break something. If it does, there's really no easy way to get this fixed before v2.0.
Note: I rechecked expire-tool behavior with version 1.2.4 and it's working the same way as 1.2.6, so my problem have more long history, than I thought.
participants (3)
-
Nikita Koshikov
-
Simon Gao
-
Timo Sirainen