- dovecot - dovecot.org

[Dovecot] Corrupted mboxes with v2.2.4, posix_fallocate and GFS2
by Francesco Prelz 05 Aug '13

05 Aug '13

Hi, on a clustered Dovecot server installation that was recently moved from a shared GPFS filesystem to GFS2, occasional corruptions in the users' INBOXes started appearing, where a new incoming message would be appended directly after a block of NUL bytes, and be scanned by dovecot as being glued to the preceding message. I traced this to the file extension operation performed in mbox_sync_handle_eof_updates, where the 'file_set_size' call is used. If available, file_set_size will use the posix_fallocate call. In GFS2 posix_fallocate increases the file size in 4 kB chunks (there seems to be no guarantee anyway that posix_allocate will extend a file by the exact size requested). After a successful posix_fallocate call, mbox_sync_handle_eof_updates currently proceeds in rewriting the mailbox starting from the originally intended 'file_size': 1306 if (file_set_size(sync_ctx->write_fd, 1307 file_size + -sync_ctx->space_diff) < 0) { 1308 mbox_set_syscall_error(sync_ctx->mbox, 1309 "file_set_size()"); 1310 if (ftruncate(sync_ctx->write_fd, file_size) < 0) { 1311 mbox_set_syscall_error(sync_ctx->mbox, 1312 "ftruncate()"); 1313 } 1314 return -1; 1315 } 1316 mbox_sync_file_updated(sync_ctx, FALSE); 1317 1318 if (mbox_sync_rewrite(sync_ctx, mail_ctx, file_size, 1319 -sync_ctx->space_diff, padding, 1320 sync_ctx->need_space_seq, 1321 sync_ctx->seq) < 0) 1322 return -1; When posix_fallocate extends the mailbox beyond the requested 'file_size', a variable size block of NUL bytes is left behind at the tail of the mailbox, with the side effects described above. I successfully worked around this issue by undefining HAVE_POSIX_FALLOCATE, as the performance penalty with falling back to direct block appends seems small. At least a size check (and possible truncation) after the mbox_sync_file_updated call above should probably be added. I thought that the issue would be anyway worth bringing to your attention. Thanks. Francesco Prelz INFN - Sezione di Milano

2 2

[Dovecot] problems with doveadm
by listserv 05 Aug '13

05 Aug '13

Hello, since few weeks i have problems with the command: doveadm expunge -A mailbox Trash savedbefore 7d => kernel: doveadm[11609]: segfault at 10 ip b75f146a sp bf856ad0 error 4 in libdovecot.so.0.0.0[b7585000+c8000] but for a single user it's works fine: doveadm expunge -u test(a)vtlx.de mailbox Trash savedbefore 30h doveconf -n # 2.2.4 (f16816a69130): /etc/dovecot/dovecot.conf # OS: Linux 3.2.45.stk32 i686 Debian jessie/sid ext3 I hope there is a solution for this. with regards Mario -- 中華人民共和國

3 8

[Dovecot] attachments not with email causing FETCH BODY[] failed
by Anand Kumria 05 Aug '13

05 Aug '13

Hi, Anyone else experiencing this (Dovecot 2.2.4, attachments stored separately): dovecot: imap(user(a)kamdha.com): Error: file_istream.open(/home/ example.com/user/attachments/f5/f0/f5f0f2c08c4311fa404d090a703c3b492f2ea718…) failed: No such file or directory dovecot: imap(user(a)example.com): Error: read(BODY[]) failed: No such file or directory (FETCH for mailbox INBOX UID 42501) dovecot: imap(user(a)example.com): Disconnected: FETCH failed in=186 out=86389 My dovecot.conf attachment related config is: mail_attachment_dir = /home/%d/%n/attachments mail_attachment_min_size = 128k mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} So, questions: - anyone else experiencing this? - how might this occurred? - what is the best way to find the corrupted message? - how should I go about fixing this? Any help or pointers would be appreciated. Thanks, Anand

3 4

[Dovecot] Passing data safely in password_key?
by Attila Nagy 05 Aug '13

05 Aug '13

Hi, I would like to convert my custom POP/IMAP proxy to Dovecot's. In this proxy I do more than giving back user name, password and the host and I need extra information. Luckily all of them are available as variables, but more than one comes as user input (like user name and cleartext password) and I'm not sure how to pass them safely. Obviously I would need a separator, which is guaranteed not to show up either in user name and the cleartext password. Should I use escape (%E) here, or is there a better way?

2 4

[Dovecot] ACL not copied to new mailboxes
by Markus Weippert 05 Aug '13

05 Aug '13

Hi, I had some trouble with ACLs not being copied to new child mailboxes in dovecot 2.2.4. It doesn't matter whether it's a private, shared or public mailbox I create the folder in, the ACLs just seems to be ignored. Anyway, I debugged the executable and came up with the attached patch. Since I'm not a developer, I'm not 100% sure about this but it works for me now. Regards, Markus # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql # doveconf -n # 2.2.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-48-generic x86_64 Ubuntu 12.04.2 LTS auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_debug = yes auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes first_valid_gid = 122 first_valid_uid = 114 imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags last_valid_gid = 122 last_valid_uid = 114 mail_debug = yes mail_gid = vmail mail_plugins = acl mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace { list = children location = maildir:/var/vmail/public/%d:LAYOUT=fs prefix = Public/ separator = / subscriptions = no type = public } namespace { list = children location = maildir:/var/vmail/mail/%%d/%%n:INDEX=~/shared/%%d/%%n prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/auth/%s.master driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/var/vmail/acls:cache_secs=300 acl_shared_dict = file:/var/vmail/mail/shared-mailboxes.db sieve = ~/dovecot.sieve sieve_after = /var/vmail/sieve/after sieve_before = /var/vmail/sieve/before sieve_default = /var/vmail/sieve/default.sieve sieve_dir = ~/sieve sieve_global_dir = /var/vmail/sieve/global } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0777 } } service imap-login { process_min_avail = 10 service_count = 0 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/dovecot.pem ssl_cipher_list = HIGH:!MD5:!eNULL ssl_key = </etc/ssl/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = acl sieve } protocol lda { mail_plugins = acl sieve } protocol imap { mail_plugins = acl imap_acl }

2 1

[Dovecot] Maildir Synchronization warnings
by Kavish Karkera 05 Aug '13

05 Aug '13

Hi, We are repeatedly getting these below warnings for some of our users, al though we have no complaints from them yet, we need to know why these warning occurs. So it would be help full if some one explain these warning msg in detail. ----------------------------------------------------------------------------------------------------------------------------------------------------- Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 94 seconds (23191 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 94 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 154 seconds (43129 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 162 seconds (43129 readdir()s, 0 rename()s to cur/, why=0xc) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 162 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Locking transaction log file /indexes//mail/v3store/example.com/kavish.karkera(a)example.com/.INBOX/dovecot.index.log took 50 seconds ----------------------------------------------------------------------------------------------------------------------------------------------------- Thanks In advance, Regards, Kavish Karkera

4 6

[Dovecot] [Metadata Plugin] Q: Source Repository
by Christoph Hintermüller 05 Aug '13

05 Aug '13

Hi I'm currently trying to package V14 of Metadata plugin for Ubuntu Quantal (https://launchpad.net/~privatedata/+archive/dovecot-2.2.2/) to be installed along with dovecot 2.2.4. According to my efforts packaging the plugin i do have some questions For future releases, is it possible to add a link to the current release on the meta data mercurial page which includes in the filename of the tar.gz archive the current version number of the plugin ( metadata-plugin-v14.tar.gz). That would allow to add an appropriate line in the debian/watch file to detect when a new release of the plugin is available. Currently i have to check mercurial page manually and use an updated version mangling entry to ensure that the packaging system downloads the latest version. The link would help to simplify this. Especially as the base link in my debian/watch file currently points to tip.tar.gz and not to the archive the linke of the V14 tag points to. The sources i got represent the tip.tar.gz downloaded on July 19th 2013 23:16 CET which likely do not exactly correspond to the contents of the archive containing sources of V14. Therefore a versioned link to the source archives of the latest version would simplify things a lot, especially adding patches released for the latest version. As well as reporting the following addition. Further I have added a doc directory stub, containing the configuration examples found in thread http://dovecot.2317879.n4.nabble.com/dovecot-metadata-8-released-tp10091.ht… Greetings Christoph

1 0

[Dovecot] Subfolders of Inbox with LAYOUT=fs
by Keith Edmunds 05 Aug '13

05 Aug '13

We've just converted to LAYOUT=fs. Some users had subfolders under Inbox, and for those users some MUAs no longer show those folders. The directory structure is that the cur, tmp and new folders are in the 'location' directory, but the subfolders are in $location/INBOX/$foldername. I am suspicious of this: 12 list "" INBOX * LIST (\HasNoChildren) "/" "INBOX" 12 OK List completed. I have tried moving the cur, tmp and new folders to $location/INBOX, and changing 'location' in dovecot.conf, but the subfolders still do not show. Question: is it legitimate to have subfolders of INBOX when using LAYOUT=fs? If so, what am I missing? Thanks, Keith Config: # dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1 ext3 lda_mailbox_autocreate = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_home = /home/%u mail_location = maildir:/home/imapmail/%u:LAYOUT=fs mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/home/imapmail/%u:LAYOUT=fs prefix = separator = / type = private } namespace { hidden = no list = children location = maildir:/home/imapmail/public:INDEX=/home/imapmail/public.indexes/%u.index:LAYOUT=fs prefix = public/ separator = / subscriptions = no type = public } passdb { driver = pam } plugin { acl = vfile acl_shared_dict = file:/home/imapmail/shared-mailboxes fts = squat home = /home/%u sieve = /home/sieve/%u.sieve sieve_dir = /home/sieve/%u } protocols = imap sieve service auth { user = root } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/dovecot/TCL_IMAPS_server.crt ssl_key = </etc/dovecot/TCL_IMAPS_server.key userdb { args = uid=imapmail gid=imapmail driver = static } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = acl imap_acl fts fts_squat } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve } -- "You can't live a perfect day without doing something for someone who will never be able to repay you."

2 2

[Dovecot] Help needed with SOLR integration
by Nigel Smith 05 Aug '13

05 Aug '13

Hi, Is there anything wrong with this config for getting SOLR working ? I'm on Ubuntu 12.04LTS and using the bundled Jetty and SOLR. Calling "http://localhost:8080/solr/update?optimize=true" doesn't return an error. And I'm pretty sure I've replaced the schema ? Help & thanks in advance Nigel # 2.0.19: /etc/dovecot/dovecot.conf # OS: Linux 3.5.0-37-generic x86_64 Ubuntu 12.04.2 LTS mail_location = maildir:~/Maildir managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii -numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave passdb { args = scheme=CRYPT username_format=%u /etc/dovecot/users driver = passwd-file } plugin { fts = solr fts_solr = url=http://localhost:8080/solr/ sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = its-virtmail user = its-virtmail } } service imap-login { process_min_avail = 3 } service lmtp { process_min_avail = 5 unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0600 user = postfix } user = its-virtmail } service managesieve-login { inet_listener sieve { port = 4190 } process_min_avail = 3 } ssl = required ssl_cert = </etc/ssl/certs/mx_example_com.pem ssl_cipher_list = TLSv1+HIGH:!SSLv2:!aNULL:!eNULL:!3DES:!PSK:@STRENGTH ssl_key = </etc/ssl/private/mx_example_com.key userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol imap { imap_client_workarounds = delay-newmail mail_max_userip_connections = 10 mail_plugins = fts fts_solr } protocol pop3 { mail_max_userip_connections = 10 pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address = postmaster quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n%r } protocol lmtp { mail_plugins = sieve postmaster_address = bp(a)example.com }

1 0

[Dovecot] Weird log entries when some users are composing messages in Thunderbird
by Charles Marcus 05 Aug '13

05 Aug '13

Hi all/Timo, When some of our users are composing a message in Thunderbird, I see a ton of these in the logs, and they keep repeating the entire time they are typing until the message gets sent... > 2013-08-05T10:34:42-04:00 myhost dovecot: imap-login: Login: user=<user(a)media-brokers.com>, method=PLAIN, rip=###.###.###.###, lport=993, mpid=8152, TLS, session=<El2GNTTjNgDAqAIc> > 2013-08-05T10:34:42-04:00 myhost dovecot: imap(user(a)media-brokers.com): Disconnected: Disconnected in IDLE in=149 out=12449 > 2013-08-05T10:34:42-04:00 myhost dovecot: imap-login: Login: user=<user(a)media-brokers.com>, method=PLAIN, rip=###.###.###.###, lport=993, mpid=8153, TLS, session=<lJedNTTjNwDAqAIc> > 2013-08-05T10:34:42-04:00 myhost dovecot: imap(user(a)media-brokers.com): Disconnected: Disconnected in IDLE in=149 out=12449 I confirmed that their 'Auto-Save' settings are the defaults (5 minutes, same as mine), and when I'm composing a message, there are no entries like this. The only other option I could think of that might affect t his is the 'spell check as you type', which is also enabled on mine. Also - I'm not sure this *always* happens to these users... all I know is, I see these flooding the logs sometimes, and when I remote in, that user is composing a message. Anyone have any idea why this happens to only a few of our users, and possibly only sometimes? Could it be something in the HTML of the quoted text when they are replying to certain messages? Thanks, -- Best regards, Charles

2 4