- dovecot - dovecot.org

[Dovecot] Multi Server Exim/Dovecot and NFS
by Robert Blayzor 16 Jul '12

16 Jul '12

In reading the Dovecot NFS WIKI entry: http://wiki2.dovecot.org/NFS It mentions a FreeBSD NFS client caching bug. I believe that bug only exists pre-8.x ? It seems that a patch in the PR notes was MFC'd and made it into 8.x at some point. (at least I checked the source in 8.3 and it was there). So I assume that FreeBSD 8.x mitigates the issue? Maybe not? So our deployment is multi server Dovecot/Exim running on FreeBSD servers, frontended by a layer3 load balancing switch and a NetApp NFSv3 datastore. The only real concern we have right now is race conditions and file write contention on the Dovecot index files. Our original plan was to use Dovecot LMTP on the servers to deliver to the inbox, but that doesn't seem to be recommended as we'll have a number of servers all acting as primary MX for domains and delivering to the same set of mailboxes. So are the Dovecot dotlock file locking mechanisms not enough for multiple servers trying to write/access index files in Maildirs? If we have to deliver straight to the ~/Maildir/new out of Exim, that's fine, also. We're just wondering if there is a added benefit of actually using LMTP. (other than making sure the index are kept the most recent). The other concern we have is client mailbox access. While we can make sessions "sticky" from one IP so they hit the same server, it doesn't help with the person that checks their email via POP from their PC while having their mobile phone access it via IMAP. (obviously from different IP addresses). So I guess there is some possibly index file write contention there as well. (though hopefully this should be rare). I'm curious as to know others experiences with a similar deployment. TIA -- Robert Blayzor INOC, LLC rblayzor(a)inoc.net http://www.inoc.net/~rblayzor/

5 7

[Dovecot] ACL with MySQL dict => full table scans
by Roland Rosenfeld 16 Jul '12

16 Jul '12

Hi! I'm just installing ACL support on 2.1.6. When a user changes these ACLs, this results in a full table scan on the dictionary, that tells dovecot who shares with whom. Here's what queries I see when I run $ doveadm acl delete -u user1 shared/user2/folder user=user1 1: SELECT dummy,to_user,from_user FROM imap_acl_shares 2: SELECT dummy,from_user FROM imap_acl_anyone 3: BEGIN 4: DELETE FROM imap_acl_shares WHERE to_user = 'user1' AND from_user = 'user1' 5: INSERT INTO imap_acl_shares (dummy,to_user,from_user) VALUES ('1','user1','user1') ON DUPLICATE KEY UPDATE dummy='1' 6: DELETE FROM imap_acl_shares WHERE to_user = 'user2' AND from_user = 'user1' 7: INSERT INTO imap_acl_shares (dummy,to_user,from_user) VALUES ('1','user2','user1') ON DUPLICATE KEY UPDATE dummy='1' 8: COMMIT Line 2 is okay, since imap_acl_anyone is always empty (I'm not able to disable this query, otherwise dovecot spawns errors about a missing map, but I can live with this). Don't ask me what lines 4/5 are repeated in 6/7, but these use indexes and I can live with this. But what the hell is line 1? Why doesn't dovecot use a WHERE clause here with either to_user='user1', to_user='user2', from_user='user1', or from_user='user2'? All these WHERE clauses would use an index and speedup the query. Without the WHERE clause I expect 100000 or more results on our production system, which could slow down it massively. Here's my setup: /etc/dovecot/dovecot-dict-sql.conf: connect = host=foo dbname=mail user=imap password=hidden map { pattern = shared/shared-boxes/user/$to/$from table = imap_acl_shares value_field = dummy fields { from_user = $from to_user = $to } } map { pattern = shared/shared-boxes/anyone/$from table = imap_acl_anyone value_field = dummy fields { from_user = $from } } And dovecot -n: # 2.1.6: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-5-amd64 x86_64 Debian 6.0.5 auth_cache_negative_ttl = 5 mins auth_cache_size = 100 M auth_cache_ttl = 5 mins auth_default_realm = example.com auth_master_user_separator = * auth_mechanisms = plain login auth_verbose = yes dict { acl = mysql:/etc/dovecot/dovecot-dict-sql.conf } disable_plaintext_auth = no listen = * log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_gid = 999 mail_location = maildir:~/mail mail_plugins = acl quota mail_uid = 999 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags mmap_disable = yes namespace { list = children location = maildir:%%h/mail:INDEX=~/mail/shared/%%u prefix = shared/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = prefix = separator = / type = private } passdb { args = /etc/dovecot/passwd.master driver = passwd-file master = yes pass = yes } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl = vfile acl_shared_dict = proxy::acl quota = maildir sieve = ~/.dovecot.sieve sieve_dir = ~/mail/sieve sieve_extensions = +imapflags } protocols = imap pop3 sieve service auth { unix_listener auth-master { group = vmail mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap { process_limit = 4000 } service managesieve-login { inet_listener sieve { port = 4190 } } service managesieve { process_limit = 100 } service pop3 { process_limit = 1000 } shutdown_clients = no ssl = no syslog_facility = local2 userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 10 mail_plugins = quota imap_quota acl imap_acl } protocol pop3 { mail_plugins = quota pop3_no_flag_updates = yes pop3_uidl_format = %v.%u } Tschoeeee Roland

1 0

Re: [Dovecot] Maildir messages
by Nick＠5sg.eu 16 Jul '12

16 Jul '12

Hi Wojciech Thanks for your comeback.. > maybe - just as in article below > http://www.bishnet.net/tim/blog/tag/maildir-to-mbox/ > you like archive in such format I'm trying to switch dovecot to MBox, not just make archives. BTW: I use http://yergler.net/projects/one-off/maildir-to-mbox as it's much smaller and very simple to use to make archives. Does anyone else know the correct way to either switch the existing Maildir multiple file format to the single MBox file format. Failing that, though it will mean a LOT of work, how do I get dovecot to work in future in MBox single file format. I've entered 'mail_location = mbox:~/mail:INBOX=/%h/%u' in the 'dovecot.conf', but it still saves each message in a file of its own in the /new /cur /tmp directories ?? -------Original question -------- > Can someone guide me on changing the Maildir messages files > back to the single file format I used to have instead of the > dovecot.index /cur /tmp /new format my newer server currently > has. I'm running dovecot v1.2.9 postfix v2.7 on Ubuntu v10.4 Thanks, Nick. . . . . -- _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ . . . . . . . . ...---... (Stamp Out Spam and Save Our Servers) http://sosNow.ws -=[ do you know morse code ? ]=- . . . . . . . . _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/

2 1

[Dovecot] director: non standart ports at backends
by Костырев Александр Алексеевич 15 Jul '12

15 Jul '12

hello, I'm trying to figure out how to proxy pop3 and pop3s that listens on non-standart ports at backends. For example, pop3 is at 1110 and pop3s at 1995 (on backend side). is it possible? how should I separate this ports in director's config? it's easy for one port: for example lmtp - you just use passdb in protocol lmtp {}

3 5

[Dovecot] Issues with Plus addressing and detail part of subaddress.
by Sean Kamath 15 Jul '12

15 Jul '12

Hi all, So, it's been a while. I'm just ramping back up with Dovecot after a hiatus of a few years. I've been using it fine, complete with Sieve, for about a year, with little or no issues. I finally got around to setting up a second account, to start figuring out how I want to do some plus address filtering. In a nutshell, I want to put mail for foo+bar in foo's 'bar' folder. That's in the wiki. I did this: require ["fileinto", "envelope", "subaddress"]; if header :contains "subject" ["test"] { fileinto "Test"; } if envelope :detail "to" "test" { fileinto "test"; stop; } So, this fails miserably. No error message, nothing in the logs, no nothing. Yet, when I run sieve-test .dovecot.sieve /path/to/testmessage it works just fine. I even run sieve-test -e .dovecot.sieve /path/to/testmessage and it files it just fine into the folder. I tried adding some debug logging, but it looks like vnd.dovecot.debug isn't in my version. I searched the archives, and found a bit about adding (explicitly) the delimiter to the plugin section of 90-sieve.conf. So I did that. I'm running 2.0.13 of dovecot, and pigeonhole 0.2.3 on Solaris 10. Before I dive in and upgrade everything, am I missing something simple? The "test" bit works just fine (and in another account as well), so I think something's just not seeing the detail in subaddressing when mail is delivered (even though it works just fine with sieve-test). Thanks, Sean

3 3

[Dovecot] Migrating with dsync and INBOX.INBOX namespace issue
by Nikolai Schupbach 15 Jul '12

15 Jul '12

I'm working on migrating IMAP users from Mirapoint Message Servers to Dovecot servers. The Mirapoint IMAP server uses INBOX as a namespace for private mailboxes. I created the same namespace in Dovecot, but when I migrate mailboxes dsync seems to ignore/miss the namespace on Mirapoint and I end up with all mailboxes under a INBOX.INBOX hierarchy. Mirapoint Source Server ----------------------- * OK server.name Mirapoint IMAP4 3.8.3-GA server ready 1 LOGIN test(a)test.com test 1 OK User logged in 2 NAMESPACE * NAMESPACE (("INBOX." ".")) (("user." ".")) (("" ".")) 2 OK Completed 3 LIST "" * * LIST () "." "INBOX" * LIST () "." "INBOX.Draft" * LIST () "." "INBOX.Sent" * LIST () "." "INBOX.Trash" 3 OK Completed Running Dsync ------------- # doveadm -v -o imapc_host=server.name imapc_user=test(a)test.com -o imapc_password=test backup -R -u test(a)test.com imapc: dsync(test(a)test.com): Info: INBOX.INBOX.Sent: only in source (guid=7d3c7eaa71cdf47ee8a1192687cda8cd) dsync(test(a)test.com): Info: INBOX.INBOX.Draft: only in source (guid=a6a03ff3d6ec9066b46013f4affb109a) dsync(test(a)test.com): Info: INBOX.INBOX.Trash: only in source (guid=b36f688eae08506001091708006abe5a) dsync(test(a)test.com): Info: INBOX: only in source (guid=c92f64f79f0d1ed01e6d5b314f04886c) Dovecot Destination Server Post Dsync ------------------------------------- * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. 1 LOGIN test(a)test.com test 1 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS MULTIAPPEND UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS SPECIAL-USE QUOTA ACL RIGHTS=texk] Logged in 2 NAMESPACE * NAMESPACE (("INBOX." ".")) NIL NIL 2 OK Namespace completed. 3 LIST "" * * LIST (\HasChildren) "." "INBOX" * LIST (\HasNoChildren) "." "INBOX.INBOX.Sent" * LIST (\HasNoChildren) "." "INBOX.INBOX.Trash" * LIST (\HasNoChildren) "." "INBOX.INBOX.Draft" 3 OK List completed. I noticed that Mirapoint IMAP also advertises a third empty namespace and I was wondering if this was causing it, but even if I explicitly set -n "INBOX." I still have the same issue. Below is my doveconf -n output: # 2.1.6: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-RELEASE-p3 amd64 auth_cache_negative_ttl = 0 auth_verbose = yes default_vsz_limit = 512 M deliver_log_format = subject=%s msgid=%m: %$ disable_plaintext_auth = no imapc_features = rfc822.size mail_gid = vmail mail_home = /mailstore/domains/%d/%n mail_location = mdbox:~/mdbox mail_plugins = quota acl mail_log notify mail_temp_dir = /var/tmp mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave spamtest namespace { inbox = yes location = prefix = INBOX. separator = . } passdb { args = /usr/local/etc/dovecot/dovecot-ldap-passdb.conf driver = ldap } plugin { acl = vfile:/usr/local/etc/dovecot/global-acls mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = dict:User quota::file:%h/.dovecot-quota quota_warning = storage=95%% quota-warning 95 %u sieve = ~/.dovecot.sieve sieve_extensions = +spamtest sieve_max_redirects = 32 sieve_spamtest_max_value = 10 sieve_spamtest_status_header = X-Spam-Score sieve_spamtest_status_type = score } service auth-worker { user = $default_internal_user } service auth { client_limit = 5420 unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } user = $default_internal_user } service imap-login { process_min_avail = 12 service_count = 0 vsz_limit = 512 M } service imap { process_limit = 4096 vsz_limit = 512 M } service lmtp { process_min_avail = 12 user = vmail } service pop3-login { process_min_avail = 12 service_count = 0 vsz_limit = 512 M } service pop3 { process_limit = 1024 vsz_limit = 512 M } service quota-warning { executable = script /usr/local/bin/quota-warning unix_listener quota-warning { user = vmail } user = vmail } ssl = no userdb { args = /usr/local/etc/dovecot/dovecot-ldap-userdb.conf driver = ldap } verbose_proctitle = yes protocol imap { mail_max_userip_connections = 50 mail_plugins = quota acl mail_log notify imap_quota imap_acl } protocol lmtp { mail_plugins = quota acl mail_log notify sieve } Any help would be much appreciated. Regards, Nikolai.

1 1

[Dovecot] Maildir messages
by Nick＠5sg.eu 13 Jul '12

13 Jul '12

Hi all, Hopefully, I'm sending this to the right place... Can someone guide me on changing the Maildir messages files back to the single file format I used to have instead of the dovecot.index /cur /tmp /new format my newer server currently has. I'm running dovecot v1.2.9 postfix v2.7 on Ubuntu v10.4 Thanks, Nick. . . . . MU/Webcasting -- _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ . . . . . . . . Please join my Tech MeetUp group that meets in the NYC metro area Webcasting and Video Streaming Network: http://TheWebcasting.net . . . . . . . . _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/ _/

7 13

[Dovecot] imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument
by Angel L. Mateo 13 Jul '12

13 Jul '12

Hello, More problems (it seems that we won't be able to put our new servers in production). The problem now is that when we reach 1000 dovecot processes we have errors like: Jul 9 12:10:22 myotis31 dovecot: imap-login: Panic: epoll_ctl(add, 6) failed: Invalid argument Jul 9 12:10:22 myotis31 dovecot: imap-login: Fatal: master: service(imap-login): child 32251 killed with signal 6 (core dumps disabled) and server doesn't accept any new connection. Our servers are running in ubuntu 12.04 with kernel 3.2.0-24-generic kernel. I have found a similar problem at https://bugzilla.redhat.com/show_bug.cgi?id=681578 but it is for redhat kernel of series 2.6. Could be the same problem? Any pointer to this problem for ubuntu kernels? Or for generic 3.0 kernels? Any idea? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868887590 Fax: 868888337

3 5

[Dovecot] mail_log plugin logs UTF-8 mailbox name
by Ewald Dieterich 11 Jul '12

11 Jul '12

A small bug in the mail_log plugin: When I copy a mail from the inbox to a mailbox with special characters, the mailbox name is logged in UTF-8: [...] copy from INBOX: box=INBOX.Euro €, [...] I guess the name should be in mUTF-7, like it is done when I copy the other way round: [...] copy from Euro &IKw-: box=INBOX, [...]

2 2

[Dovecot] more info (more dsync problems)
by Jeff Gustafson 10 Jul '12

10 Jul '12

Rerunning the dsync command right after the timeout (see previous message) gave me these errors: dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user%domain.com/mdbox/s torage/m.60 (around offset=807789): msg header has bad magic value dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user%domain.com/mdbox/s torage/m.60 (around offset=807789): msg header has bad magic value dsync-local(vmail): Error: remote: dsync-remote(vmail): Warning: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: Inconsistency in map index (2,75904 != 2,82524) dsync-remote(vmail): Warning: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: rebuilding indexes dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage/m.60 (around offset=807789): msg header has bad magic value dsync-local(vmail): Error: remote: dsync-remote(vmail): Warning: dbox: Copy of the broken file saved to /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage/m.60.broken dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: Corrupted dbox file /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage/m.60 (around offset=807789): EOF reading msg header (got 0/30 bytes) dsync-local(vmail): Error: remote: dsync-remote(vmail): Warning: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: rebuilding indexes dsync-local(vmail): Error: remote: dsync-remote(vmail): Error: mdbox /home/.incoming_mail_migrations/users/user% domain.com/mdbox/storage: Duplicate GUID 1341304555.M433317P26661.n13,S=142447,W=144391 in m.62:790375 and m.60:790375

1 0