- dovecot - dovecot.org

[Dovecot] dsync on large mailbox "fails"
by Jeff Kletsky 25 Apr '12

25 Apr '12

I've been trying to convert an existing set of mbox mail to maildir using dsync, but it seems to fail when it gets to the large boxes. $ dsync -Dv mirror maildir:~/Maildir Everything seems to work reasonably well until the first of the large boxes hits: dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: Synchronization took 1210 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for 1210 seconds After this, the doveadm process goes from ~75% of the CPU core utilization to 100% and stays there, even overnight. I've also tried $ dsync -Dv backup maildir:~/Maildir with similar results. While I did see some threads on dsync speed, they seemed to be minutes for a few GB of data, not hours. There is plenty of space on the drive, and, from what I understand, there is not a file-count limit on FreeBSD's ufs (at least not one that I'm anywhere near) Any suggestions? Thanks! Jeff [jeff@mail ~]$ du -h -d 0 ~/mail 3.9G /home/jeff/mail [jeff@mail ~]$ ls -lh /var/mail/jeff -rw------- 1 jeff jeff 554M Apr 23 11:27 /var/mail/jeff [jeff@mail ~]$ df -h Filesystem Size Used Avail Capacity Mounted on /dev/ada0s2g 38G 18G 16G 53% / (Note that the hostname and the location of the ssl certificate has been manually edited for this message in the following output) From dmesg.boot: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-STABLE #0: Sat Apr 21 17:13:07 PDT 2012 root@my.host.name:/usr/obj/usr/src/sys/GENERIC i386 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Family = 6 Model = 1c Stepping = 2 Features=0xbfe9fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE> AMD Features=0x20100000<NX,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2077757440 (1981 MB) [jeff@mail ~]$ doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 listen = * mail_location = mbox:~/mail/:INBOX=/var/mail/%u mail_privileged_group = mail namespace { inbox = yes location = prefix = separator = / } passdb { args = session=yes dovecot driver = pam } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 200 } ssl = required ssl_cert = </path/to/certificate.pem ssl_key = </path/to/certificate.pem userdb { args = blocking=yes driver = passwd } verbose_proctitle = yes protocol imap { imap_client_workarounds = tb-extra-mailbox-sep }

2 5

[Dovecot] trash plugin bug?
by Cor Bosman 25 Apr '12

25 Apr '12

Hey all, has anyone ever tried turning on the trash plugin when the directory is already over quota? I see some messages being deleted, but it seems it just deletes enough to fit the new email, not enough to go below quota. Example: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591036 5242880) * QUOTA "Spam Quota" (STORAGE 21173 20480) I now move a 5K message into this folder, and I see: Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226337, msgid=<3YVKIIE4II6R0064BA7IM55C2Q4MD01VX12U1I(a)shedevils.com>, size=12984 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): expunge: box=Spam, uid=226338, msgid=<aab6bffa3ad83767d4a7bc571c31449c(a)bonus-news.com>, size=5623 Apr 23 22:26:43 lmtp1 dovecot: imap(cor): save: box=Spam, uid=227759, msgid=<20120423191509.F3B584A355B(a)signal.groundspeak.com>, size=2694 And now: . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam Quota" * QUOTA "User quota" (STORAGE 591016 5242880) * QUOTA "Spam Quota" (STORAGE 21158 20480) . OK Getquotaroot completed. This seems like a possible bug. For one, it shouldnt be allowed to actually save a message into that folder, but it does. The email does end up in the Spam folder even though the folder is over quota. And of course it doesnt clear up enough space to have the new email fit, it just deleted enough to match the 5K. Without the trash plugin quota is being properly enforced. Everything works fine if you start out below the quota limit btw. Regards, Cor ps: this is 2.1.4, but with the recent trash and quota patches applied.

2 3

[Dovecot] sievec segfaults
by interfaSys sàrl 24 Apr '12

24 Apr '12

Hello, sievec segfaults when it reads a file. I've tried with both the standard and gold linker. I've tried with various sieve files and they all make it crash. I've compiled pigeonhole with -ggdb, but I'm not getting much info Program received signal SIGSEGV, Segmentation fault. 0x0000000800b8e587 in strlen () from /lib/libc.so.7 (gdb) bt full #0 0x0000000800b8e587 in strlen () from /lib/libc.so.7 No symbol table info available. #1 0x0000000800a12e8c in str_append () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x000000080077f02d in sieve_binary_save () from /usr/lib/dovecot/libdovecot-sieve.so.0 No symbol table info available. #3 0x0000000000605bf0 in environ () No symbol table info available. #4 0x0000000000000000 in ?? () No symbol table info available. #5 0x00007fffffffeb6e in ?? () No symbol table info available. #6 0x0000000000000001 in ?? () No symbol table info available. #7 0x0000000000000000 in ?? () No symbol table info available. #8 0x000000080060878d in ?? () from /libexec/ld-elf.so.1 No symbol table info available. #9 0x0000000000605cd9 in ?? () No symbol table info available. #10 0x0000000801000ea8 in ?? () No symbol table info available. #11 0x0000000000005008 in ?? () No symbol table info available. #12 0x0000000000000000 in ?? () No symbol table info available. #13 0x0000000801099040 in ?? () No symbol table info available. #14 0x0000000000000000 in ?? () No symbol table info available. #15 0x0000000000000180 in ?? () No symbol table info available. #16 0x0000000000605bf0 in environ () No symbol table info available. #17 0x0000000000000000 in ?? () No symbol table info available. #18 0x00007fffffffeb6e in ?? () No symbol table info available. #19 0x0000000000000001 in ?? () No symbol table info available. #20 0x0000000000000000 in ?? () No symbol table info available. #21 0x0000000000000000 in ?? () No symbol table info available. #22 0x0000000000402f93 in main () No symbol table info available. Cheers, Olivier

2 5

[Dovecot] Dovecot Virtual Users with Users Domain Mapping
by Piotr Maier 24 Apr '12

24 Apr '12

I have successfully compiled, configured and ran Dovecot with virtual users feature. Here's part of my /etc/dovecot.conf configuration file: mail_location = maildir:~/Maildir auth default { mechanisms = plain login userdb passwd-file { args = /home/%d/etc/passwd } passdb passwd-file { args = /home/%d/etc/shadow } socket listen { master { path = /var/run/dovecot/auth-worker mode = 0600 } } } I faced one issue I can't resolve myself. Is there anyway to create users' domains mapping and provide username in mail_location? Example of usersdomains file: domain.com:user1 domain2.org:user3 Examples: 1. currently I have /home/domain.com/user/Maildir 2. I'd like to have /home/SYSTEM_USER/domain.com/user/Maildir Can I achieve this somehow?

1 0

[Dovecot] migration dove 2.0 2.1 shared namespace acl broken
by Robert Schetterer 24 Apr '12

24 Apr '12

Hi Tim, after upgrade to 2.1 acl with shared namespace seems to be broken it looks like depend to No lookup right to mailbox: shared/ it runs perfect under 2.0.20 -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

2 10

[Dovecot] quota not being calculated
by Cor Bosman 23 Apr '12

23 Apr '12

It looks like my quota isnt being calculated properly after I started setting quota to a specific folder. The quota in that folder is always starting out at 0, and only new email is being added to the quota. If I remove the maildirsize file, and recalculate, it still starts at 0. Once email arrives in the folder, it gets added to maildirsize. This is with maildirquota, in perhaps a bit of an exotic config. The idea is to set a specific quota to Spam folder, using the trash plugin to keep it at a certain size. . getquotaroot "Spam" * QUOTAROOT "Spam" "User quota" "Spam quota" * QUOTA "User quota" (STORAGE 40848 1228800) * QUOTA "Spam quota" (STORAGE 0 20000) . OK Getquotaroot completed. Once 1 email gets added: .Spam# cat maildirsize 20480000S 0 0 2017 1 namespace { hidden = no ignore_on_failure = no inbox = yes list = yes location = maildir:%h:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/INBOX prefix = separator = / subscriptions = yes type = private } namespace spam { hidden = yes ignore_on_failure = no inbox = no list = yes location = maildir:%h/.Spam:INDEX=/var/spool/mail/dovecot-control/indexes/%1u/%2u/%u/.Spam:CONTROL=/var/spool/mail/dovecot-control/%1u/%2u/%u/.Spam prefix = Spam/ separator = / subscriptions = no type = private } plugin { quota = fs:User quota quota2 = maildir:Spam quota:ns=Spam/ quota2_rule = *:storage=20000K sieve = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/dovecot.sieve sieve_before = /etc/sieve/before sieve_dir = /var/spool/mail/dovecot-control/sieve/%1u/%2u/%u/scripts trash = /etc/dovecot/conf.d/dovecot-trash.conf.ext } (full config: http://pastebin.com/Mui4X7Zh)

2 6

[Dovecot] virtual plugin and shared namespace
by l.chelchowski＠eurocar.pl 23 Apr '12

23 Apr '12

Hi, I have a problem with virtual plugin and geting mails form shared namespace. dovecot-virtual: ---- Shared/* Public/* * unseen ---- dovecot.conf: ---- namespace { inbox = yes location = prefix = separator = / type = private } namespace { list = children location = maildir:/home/vmail/public/:CONTROL=/var/mail/vmail/%d/%u/control/public:INDEX=/var/mail/vmail/%d/%u/index/public:LAYOUT=fs prefix = Public/ separator = / subscriptions = yes type = public } namespace { list = children location = maildir:%%h/:INDEX=/var/mail/vmail/%d/%u/index/shared/%%u prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace { list = children prefix = Virtual/ separator = / location = virtual:~/virtual } ---- When I enter the virtual mailbox Unread I get this debug info: virtual mailbox Virtual/Unread: Skipping non-existing mailbox Shared/test(a)domain.pl/ Is it a bug or misconfig? -- Łukasz Chełchowski

2 2

[Dovecot] error when using dovecot-lda as non root (vmail) user
by Andreas Helmcke 23 Apr '12

23 Apr '12

Hello, after finally having managed to setup up dovecot with virtual users and public folders I tried to get it connected to postfix. The problem is, that I can't get postix to deliver mail via dovecot-lda. After some testing it seems to come down to dovecot-lda behaving different when called as root then when called as user vmail. Basic setting: - dovecot 2.1.1 - all mailboxes owned by user vmail (using mail_uid and mail_gid) Test 1: as user root: /usr/libexec/dovecot/dovecot-lda -d ahelmcke < some_mail working as expected; some_mail gets stored in users INBOX Test 2: same command executed as user vmail. dovecot-lda terminates immediately. syslog-output with mail_debug = yes: Test 1: Apr 19 16:38:04 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace Allgemeines: type=public, prefix=Allgemeines/, sep=/, inbox=no, hidden=no, list=yes, subscriptions=no location=maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/ahelmcke/SEEN.Allgemeines Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/Allgemeines, index=/home/mail/user/ahelmcke/SEEN.Allgemeines, control=, inbox=, alt= Apr 19 16:38:04 elablnmail02 dovecot: lda(ahelmcke): Debug: none: root=, index=, control=, inbox=, alt= Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Destination address: ahelmcke(a)elablnmail02.ela-bln.local (source: user@hostname) Apr 19 16:51:37 elablnmail02 dovecot: lda(ahelmcke): msgid=unspecified: saved mail to INBOX Test 2: Apr 19 16:37:37 elablnmail02 dovecot: lda: Debug: auth input: ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Effective uid=494, gid=491, home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace inbox: type=private, prefix=INBOX/, sep=/, inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke, index=, control=, inbox=/home/mail/user/ahelmcke, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace dummy: type=private, prefix=, sep=/, inbox=no, hidden=yes, list=no, subscriptions=yes location=maildir:/home/mail/user/ahelmcke/public/ Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir++: root=/home/mail/user/ahelmcke/public, index=, control=, inbox=, alt= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: Namespace : type=private, prefix=, sep=, inbox=no, hidden=no, list=yes, subscriptions=yes location= Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: access(/home/mail/user/ahelmcke/mdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: access(/home/mail/user/ahelmcke/Maildir, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: maildir: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has .imap/: stat(/home/mail/user/ahelmcke/Mail/.imap) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has inbox: stat(/home/mail/user/ahelmcke/Mail/inbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox autodetect: has mbox: stat(/home/mail/user/ahelmcke/Mail/mbox) failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: mbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: access(/home/mail/user/ahelmcke/sdbox, rwx): failed: No such file or directory Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Debug: sdbox: couldn't find root dir Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Error: user ahelmcke: Initialization failed: Namespace '': Mail storage autodetection failed with home=/home/mail/user/ahelmcke Apr 19 16:37:37 elablnmail02 dovecot: lda(ahelmcke): Fatal: Invalid user settings. Refer to server log for more information. doveconf -n # 2.1.1: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-220.7.1.el6.x86_64 x86_64 CentOS release 6.2 (Final) auth_master_user_separator = * auth_verbose = yes first_valid_uid = 400 mail_debug = yes mail_gid = vmail mail_home = /home/mail/user/%u mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave mbox_write_locks = fcntl namespace Allgemeines { location = maildir:/home/mail/Allgemeines:INDEX=/home/mail/user/%u/SEEN.Allgemeines prefix = Allgemeines/ separator = / subscriptions = no type = public } namespace dummy { alias_for = INBOX/ hidden = yes list = no location = maildir:/home/mail/user/%u/public/ prefix = separator = / type = private } namespace inbox { inbox = yes location = maildir:/home/mail/user/%u prefix = INBOX/ separator = / type = private } passdb { args = /etc/dovecot/passwd.masterusers driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } plugin { acl = vfile sieve = /home/mail/user/%u/sieve.active sieve_dir = /home/mail/user/%u/sieve.scripts sieve_global_dir = /var/lib/dovecot/sieve/global/ sieve_global_path = /var/lib/dovecot/sieve/default.sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/auth { mode = 0666 } unix_listener auth-userdb { mode = 0600 user = vmail } } service imap-login { inet_listener imap { address = * } inet_listener imaps { address = * } } service managesieve-login { inet_listener sieve { address = * port = 4190 } } ssl = required ssl_cert = </etc/pki/dovecot/certs/mail.ela-soft.com.crt ssl_key = </etc/pki/dovecot/private/mail.ela-soft.com.2048bit.nopp.key userdb { args = /etc/dovecot/dovecot-ldap.conf driver = ldap } protocol lmtp { mail_plugins = " sieve" } Any ideas? Greetings Andreas

2 8

[Dovecot] questions about SPECIAL-USE and virtual
by l.chelchowski＠eurocar.pl 23 Apr '12

23 Apr '12

Hi, I have some questions about Special-Use and virtual namespace: I want to create a virtual namespace with mailboxes: Sent, Unread, Flagged, All. 1. Is it enough to put in the config file: mailbox virtual/Sent { auto = subscribe special_use = \Sent } mailbox virtual/Unread { auto = subscribe special_use = \Unread #Is it correct? } mailbox virtual/Flagged { auto = subscribe special_use = \Flagged } mailbox virtual/All { auto = subscribe special_use = \All } and it would displays mail from every namespace I have subscribed (Shared, Public)? OR I have to make also dovecot-virtual files? 2. In dovecot-virtual file can I put for mailbox virtual/Sent something like: * \Sent 3. Is the http://wiki2.dovecot.org/Plugins/Virtual full documented, if not , where can I find it? 4. Is there a documentation for Special-Use? -- Łukasz

3 3

[Dovecot] 2.1.2 Corrupted squat uidlist
by Luca Palazzo 23 Apr '12

23 Apr '12

Hi Timo and All, after upgrading to 2.1.2 i'm getting a lot of these messages: Error: Corrupted squat uidlist file XXXXXX wrong indexid I did not have them before. Ideas? Luca

4 7