- dovecot - dovecot.org

[Dovecot] script to test CATENATE
by Mike Abbott 06 Aug '13

06 Aug '13

Attached please find a perl script which tests the CATENATE support in dovecot. I used this to test my CATENATE implementation a few years ago and it runs fine against dovecot in OS X Server. When run against dovecot-2.2.4 though it always fails or hangs, which in some cases means we interpreted RFCs differently and in other cases means it's finding bugs; both conditions are worthy of scrutiny. (It's random-number driven so every run is different.) A couple months ago I reported a few simple bugs which this script found and you fixed them; thanks. Then it started finding problems for which it's harder to isolate simple reproducible test cases. Vacations and other work interceded but now Apple is pleased to give you the script itself to allow you to iterate faster. Here are some examples of it running. One time only, pass the --init argument to store some template messages used by the real tests: $ ./catenate.pl --host your.test.server --user testuser --password 1234 --init connecting (imaps)... capability... logging in... deleting old templates mailbox... creating templates mailbox... append1... Append succeeded append2... Append succeeded logout... success $ ./catenate.pl --host your.test.server --user testuser --password 1234 connecting (imaps)... capability... logging in... append1... Append failed as it should have (bad url): <append1 NO [BADURL ;invalid;] Invalid IMAP URL: Unexpected IMAP URL path segment: `;invalid;'.> append2... Append failed as it should have (bad url): <append2 NO [BADURL /inbox/;uid=99999999] Message not found.> append3... Append succeeded [...] All tests passed. $ If it concludes with anything other than "All tests passed" or hangs then it found something that it didn't expect and that should be examined. Use the --verbose option to see the entire client-server conversation. We hope that you find this script helpful to harden your CATENATE code. I will be happy to answer any questions.

3 5

[Dovecot] problems with expire plugin
by Felix Rubio Dalmau 06 Aug '13

06 Aug '13

Hi all, I'm experiencing problems to set-up the expire plugin in dovecot. This is the relevant part of my dovecot configuration dict { expire = mysql:/etc/dovecot/dovecot-dict-expire.conf.ext } plugin { expire = Trash 15 Junk 15 expire_dict = proxy::expire } protocol imap { mail_plugins = quota zlib expire imap_quota } File dovecot-dict-expire.conf.ext looks like: connect = host=127.0.0.1 dbname=virtual_email user=<user> password=<passw> map { pattern = shared/expire/$user/$mailbox table = virtual_expires value_field = expire_stamp fields { username = $user mailbox = $mailbox } } and the mysql schema for table virtual_expires is: +--------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+-------+ | username | varchar(75) | NO | PRI | NULL | | | mailbox | varchar(255) | NO | PRI | NULL | | | expire_stamp | int(11) | NO | | NULL | | +--------------+--------------+------+-----+---------+-------+ However, when I delete any mail from the Inbox, the table is not updated, and there are no messages in the log, although mail_debug is set to "yes" Does anybody has any idea on how to solve this issue? Regards! Felix

3 9

[Dovecot] Corrupted mboxes with v2.2.4, posix_fallocate and GFS2
by Francesco Prelz 05 Aug '13

05 Aug '13

Hi, on a clustered Dovecot server installation that was recently moved from a shared GPFS filesystem to GFS2, occasional corruptions in the users' INBOXes started appearing, where a new incoming message would be appended directly after a block of NUL bytes, and be scanned by dovecot as being glued to the preceding message. I traced this to the file extension operation performed in mbox_sync_handle_eof_updates, where the 'file_set_size' call is used. If available, file_set_size will use the posix_fallocate call. In GFS2 posix_fallocate increases the file size in 4 kB chunks (there seems to be no guarantee anyway that posix_allocate will extend a file by the exact size requested). After a successful posix_fallocate call, mbox_sync_handle_eof_updates currently proceeds in rewriting the mailbox starting from the originally intended 'file_size': 1306 if (file_set_size(sync_ctx->write_fd, 1307 file_size + -sync_ctx->space_diff) < 0) { 1308 mbox_set_syscall_error(sync_ctx->mbox, 1309 "file_set_size()"); 1310 if (ftruncate(sync_ctx->write_fd, file_size) < 0) { 1311 mbox_set_syscall_error(sync_ctx->mbox, 1312 "ftruncate()"); 1313 } 1314 return -1; 1315 } 1316 mbox_sync_file_updated(sync_ctx, FALSE); 1317 1318 if (mbox_sync_rewrite(sync_ctx, mail_ctx, file_size, 1319 -sync_ctx->space_diff, padding, 1320 sync_ctx->need_space_seq, 1321 sync_ctx->seq) < 0) 1322 return -1; When posix_fallocate extends the mailbox beyond the requested 'file_size', a variable size block of NUL bytes is left behind at the tail of the mailbox, with the side effects described above. I successfully worked around this issue by undefining HAVE_POSIX_FALLOCATE, as the performance penalty with falling back to direct block appends seems small. At least a size check (and possible truncation) after the mbox_sync_file_updated call above should probably be added. I thought that the issue would be anyway worth bringing to your attention. Thanks. Francesco Prelz INFN - Sezione di Milano

2 2

[Dovecot] problems with doveadm
by listserv 05 Aug '13

05 Aug '13

Hello, since few weeks i have problems with the command: doveadm expunge -A mailbox Trash savedbefore 7d => kernel: doveadm[11609]: segfault at 10 ip b75f146a sp bf856ad0 error 4 in libdovecot.so.0.0.0[b7585000+c8000] but for a single user it's works fine: doveadm expunge -u test(a)vtlx.de mailbox Trash savedbefore 30h doveconf -n # 2.2.4 (f16816a69130): /etc/dovecot/dovecot.conf # OS: Linux 3.2.45.stk32 i686 Debian jessie/sid ext3 I hope there is a solution for this. with regards Mario -- 中華人民共和國

3 8

[Dovecot] attachments not with email causing FETCH BODY[] failed
by Anand Kumria 05 Aug '13

05 Aug '13

Hi, Anyone else experiencing this (Dovecot 2.2.4, attachments stored separately): dovecot: imap(user(a)kamdha.com): Error: file_istream.open(/home/ example.com/user/attachments/f5/f0/f5f0f2c08c4311fa404d090a703c3b492f2ea718…) failed: No such file or directory dovecot: imap(user(a)example.com): Error: read(BODY[]) failed: No such file or directory (FETCH for mailbox INBOX UID 42501) dovecot: imap(user(a)example.com): Disconnected: FETCH failed in=186 out=86389 My dovecot.conf attachment related config is: mail_attachment_dir = /home/%d/%n/attachments mail_attachment_min_size = 128k mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} So, questions: - anyone else experiencing this? - how might this occurred? - what is the best way to find the corrupted message? - how should I go about fixing this? Any help or pointers would be appreciated. Thanks, Anand

3 4

[Dovecot] Passing data safely in password_key?
by Attila Nagy 05 Aug '13

05 Aug '13

Hi, I would like to convert my custom POP/IMAP proxy to Dovecot's. In this proxy I do more than giving back user name, password and the host and I need extra information. Luckily all of them are available as variables, but more than one comes as user input (like user name and cleartext password) and I'm not sure how to pass them safely. Obviously I would need a separator, which is guaranteed not to show up either in user name and the cleartext password. Should I use escape (%E) here, or is there a better way?

2 4

[Dovecot] ACL not copied to new mailboxes
by Markus Weippert 05 Aug '13

05 Aug '13

Hi, I had some trouble with ACLs not being copied to new child mailboxes in dovecot 2.2.4. It doesn't matter whether it's a private, shared or public mailbox I create the folder in, the ACLs just seems to be ignored. Anyway, I debugged the executable and came up with the attached patch. Since I'm not a developer, I'm not 100% sure about this but it works for me now. Regards, Markus # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL drivers: mysql Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap nss passwd prefetch passwd-file sql # doveconf -n # 2.2.4: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 3.2.0-48-generic x86_64 Ubuntu 12.04.2 LTS auth_cache_negative_ttl = 5 mins auth_cache_size = 10 M auth_cache_ttl = 1 mins auth_debug = yes auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes first_valid_gid = 122 first_valid_uid = 114 imap_client_workarounds = tb-extra-mailbox-sep tb-lsub-flags last_valid_gid = 122 last_valid_uid = 114 mail_debug = yes mail_gid = vmail mail_plugins = acl mail_privileged_group = vmail mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { alias_for = hidden = yes inbox = no list = no location = prefix = INBOX. separator = . type = private } namespace { list = children location = maildir:/var/vmail/public/%d:LAYOUT=fs prefix = Public/ separator = / subscriptions = no type = public } namespace { list = children location = maildir:/var/vmail/mail/%%d/%%n:INDEX=~/shared/%%d/%%n prefix = Shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/auth/%s.master driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/var/vmail/acls:cache_secs=300 acl_shared_dict = file:/var/vmail/mail/shared-mailboxes.db sieve = ~/dovecot.sieve sieve_after = /var/vmail/sieve/after sieve_before = /var/vmail/sieve/before sieve_default = /var/vmail/sieve/default.sieve sieve_dir = ~/sieve sieve_global_dir = /var/vmail/sieve/global } protocols = imap lmtp sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0777 } } service imap-login { process_min_avail = 10 service_count = 0 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = </etc/ssl/dovecot.pem ssl_cipher_list = HIGH:!MD5:!eNULL ssl_key = </etc/ssl/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = acl sieve } protocol lda { mail_plugins = acl sieve } protocol imap { mail_plugins = acl imap_acl }

2 1

[Dovecot] Maildir Synchronization warnings
by Kavish Karkera 05 Aug '13

05 Aug '13

Hi, We are repeatedly getting these below warnings for some of our users, al though we have no complaints from them yet, we need to know why these warning occurs. So it would be help full if some one explain these warning msg in detail. ----------------------------------------------------------------------------------------------------------------------------------------------------- Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 94 seconds (23191 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:52:55 blade8 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 94 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 154 seconds (43129 readdir()s, 0 rename()s to cur/, why=0x1) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir: Scanning /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir/cur took 162 seconds (43129 readdir()s, 0 rename()s to cur/, why=0xc) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Maildir /mail/v3store/example.com/kavish.karkera(a)example.com/Maildir: Synchronization took 162 seconds (0 new msgs, 0 flag change attempts, 0 expunge attempts) Aug 2 12:53:54 blade6 dovecot: imap(kavish.karkera(a)example.com): Warning: Locking transaction log file /indexes//mail/v3store/example.com/kavish.karkera(a)example.com/.INBOX/dovecot.index.log took 50 seconds ----------------------------------------------------------------------------------------------------------------------------------------------------- Thanks In advance, Regards, Kavish Karkera

4 6

[Dovecot] [Metadata Plugin] Q: Source Repository
by Christoph Hintermüller 05 Aug '13

05 Aug '13

Hi I'm currently trying to package V14 of Metadata plugin for Ubuntu Quantal (https://launchpad.net/~privatedata/+archive/dovecot-2.2.2/) to be installed along with dovecot 2.2.4. According to my efforts packaging the plugin i do have some questions For future releases, is it possible to add a link to the current release on the meta data mercurial page which includes in the filename of the tar.gz archive the current version number of the plugin ( metadata-plugin-v14.tar.gz). That would allow to add an appropriate line in the debian/watch file to detect when a new release of the plugin is available. Currently i have to check mercurial page manually and use an updated version mangling entry to ensure that the packaging system downloads the latest version. The link would help to simplify this. Especially as the base link in my debian/watch file currently points to tip.tar.gz and not to the archive the linke of the V14 tag points to. The sources i got represent the tip.tar.gz downloaded on July 19th 2013 23:16 CET which likely do not exactly correspond to the contents of the archive containing sources of V14. Therefore a versioned link to the source archives of the latest version would simplify things a lot, especially adding patches released for the latest version. As well as reporting the following addition. Further I have added a doc directory stub, containing the configuration examples found in thread http://dovecot.2317879.n4.nabble.com/dovecot-metadata-8-released-tp10091.ht… Greetings Christoph

1 0

[Dovecot] Subfolders of Inbox with LAYOUT=fs
by Keith Edmunds 05 Aug '13

05 Aug '13

We've just converted to LAYOUT=fs. Some users had subfolders under Inbox, and for those users some MUAs no longer show those folders. The directory structure is that the cur, tmp and new folders are in the 'location' directory, but the subfolders are in $location/INBOX/$foldername. I am suspicious of this: 12 list "" INBOX * LIST (\HasNoChildren) "/" "INBOX" 12 OK List completed. I have tried moving the cur, tmp and new folders to $location/INBOX, and changing 'location' in dovecot.conf, but the subfolders still do not show. Question: is it legitimate to have subfolders of INBOX when using LAYOUT=fs? If so, what am I missing? Thanks, Keith Config: # dovecot -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-amd64 x86_64 Debian 7.1 ext3 lda_mailbox_autocreate = yes log_timestamp = "%Y-%m-%d %H:%M:%S " mail_home = /home/%u mail_location = maildir:/home/imapmail/%u:LAYOUT=fs mail_privileged_group = mail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace { inbox = yes location = maildir:/home/imapmail/%u:LAYOUT=fs prefix = separator = / type = private } namespace { hidden = no list = children location = maildir:/home/imapmail/public:INDEX=/home/imapmail/public.indexes/%u.index:LAYOUT=fs prefix = public/ separator = / subscriptions = no type = public } passdb { driver = pam } plugin { acl = vfile acl_shared_dict = file:/home/imapmail/shared-mailboxes fts = squat home = /home/%u sieve = /home/sieve/%u.sieve sieve_dir = /home/sieve/%u } protocols = imap sieve service auth { user = root } service managesieve-login { inet_listener sieve { port = 4190 } } ssl_cert = </etc/dovecot/TCL_IMAPS_server.crt ssl_key = </etc/dovecot/TCL_IMAPS_server.key userdb { args = uid=imapmail gid=imapmail driver = static } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = acl imap_acl fts fts_squat } protocol pop3 { pop3_uidl_format = %08Xu%08Xv } protocol lda { mail_plugins = sieve } -- "You can't live a perfect day without doing something for someone who will never be able to repay you."

2 2