- dovecot - dovecot.org

[Dovecot] Preferred LDAP Attribute for home/mail location
by Edgar Fuß 29 Jun '12

29 Jun '12

Is there, among the dovocot community, any preferred LDAP schema and attribute to use for setting the home/mail storage location? Some people seem to use the qmail schema, some a Jamm schema (whatever that is), and Markus Effinger has even created a dovecot schema (https://www.effinger.org/blog/2009/01/11/eigenes-ldap-schema-erstellen/) There may be more. I could even create my own given we have been assigned an official OID a decade ago anyway. However, sometimes I prefer to use what most other people do. I would effectively only need to store the name of the relevant NFS server.

2 1

Re: [Dovecot] RAID1+md concat+XFS as mailstorage
by Dr Josef Karthauser 29 Jun '12

29 Jun '12

Kelsey Cummings wrote: > On 06/28/12 05:56, Ed W wrote: >> So given the statistics show us that 2 disk failures are much more >> common than we expect, and that "silent corruption" is likely occurring >> within (larger) real world file stores, there really aren't many battle >> tested options that can protect against this - really only RAID6 right >> now and that has significant limitations... > > Has anyone tried or benchmarked ZFS, perhaps ZFS+NFS as backing store > for spools? Sorry if I've missed it and this has already come up. > We're using Netapp/NFS, and are likely to continue to do so but still > curious. Hi Kelsey, We're running ZFS here, and have just started using dovecot on it. No stats yet to report, but you might be interested in this edge case. One of our server started behaving badly... the database would randomly crash and not restart due to corrupted indexed. It turns out that the memory had gone bad, and that it had been bad for a while. Disk blocks were getting corrupted on read, and some on write! Luckly because we were on ZFS, which checksums all data, we were able to detect and repair most of the data (some 80mb of bad blocks distributed evenly thoughout the entire file system!) automatically, and also know exactly which files were unrecoverable (in the end just two or three files!). Also, we have hourly snapshots of all the file systems, so we were able to recover older versions of those files with minimal loss. I will never rely on a non-checksumming file system for production use again, for data that is existed to persist over time. Joe

2 1

[Dovecot] auth service: out of memory
by Mailing List SVR 29 Jun '12

29 Jun '12

Hi, I have some out of memory errors in my logs (file errors.txt attached) I'm using dovecot 2.0.19, I can see some memory leaks fix in hg after the 2.0.19 release but they seem related to imap-login service, I attached my config too, is something wrong there? Should I really increase the limit based on my settings? Can these commits fix the reported leak? http://hg.dovecot.org/dovecot-2.0/rev/6299dfb73732 http://hg.dovecot.org/dovecot-2.0/rev/67f1cef07427 Please note that the auth service is restarted when it reach the limit so no real issues, please advice thanks Nicola

2 4

[Dovecot] Removing specific entry in user/auth cache
by Angel L. Mateo 29 Jun '12

29 Jun '12

Hi, We have dovecot configured with auth cache. Is there any way to remove a specific entry (not all) from this cache? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información _o) y las Comunicaciones Aplicadas (ATICA) / \\ http://www.um.es/atica _(___V Tfo: 868887590 Fax: 868888337

4 4

[Dovecot] Integrating Dovecot with Amazon Web Services
by Gary Mort 29 Jun '12

29 Jun '12

I did some searching in the mail archives and didn't see any discussion of integration with AWS, so I wanted to through out my thoughts/plans and see if it has been done before. I am setting up my own personal website on EC2 along with an email server, and I really don't like the idea of using the disk drive as permanent mail storage. EBS is too small instance storage is ephermeral. Looking over the docs, the dbox format seems most easily copied for my needs. http://wiki2.dovecot.org/MailboxFormat/dbox To make life easy, I'll stick with just single-dbox as a start, however multi-dbox would be doable. With dbox, the only thing that I need to change is the alternate storage model: "An upshot of the way alternate storage works is that any given storage file (mailboxes/<folder>/dbox-Mails/u.* (sdbox) or storage/m.* (mdbox)) can only appear *either* in the primary storage area *or* the alternate storage area but not both — if the corresponding file appears in both areas then there is an inconsistency." First I want to add AWS S3 as a storage option for alternate storage. Then instead of the above model, the new model would be that email is always stored in alternate storage, and may be in primary storage. So, when mail comes in, I'd have Dovecot save the email to the alternate storage S3 bucket and update the indexs and other information[ideally, for convenience purposes, a few bits of relevant indexing information can be stored as metadata in the S3 object - sufficient so that instead of retrieving the entire S3 object, just the meta data can be pulled to build indexes. When a client attempts to retrieve an email message, Dovecot would check primary storage as it does now, if the message is not found than it will retrieve it from the alternate storage system AND store a copy in the primary storage. Primary storage can be periodically purged, have quota's to keep it from growing too large, etc. In this way, primary storage can be viewed as a message cache, just keeping the messages that are currently of interest, while S3 is the real data. [Ideally, this can be expanded so that when a message comes in, in addition to storing a copy in S3, an AWS SNS notification can be issued so if multiple IMAP servers are running, they can all subscribe to the same SNS channel and update themselves as needed]. This give me unlimited disk storage at S3 prices, I would even like to be able to set a few options based on the folder, so I can enable versioning on important message folders, use the even cheaper reduced redundancy storage for archives, and set expiration dates on email in the trash and spam folders so S3 will automatically purge the messages after a month. Secondly, I'd like to replace the Mysql database usage with a simpleDB database. While simpleDB lacks much of MySQL's sophistication, it doesn't seem that Dovecot is really using any of that, so simpleDB can be functionally equivalent. The primary purpose of using simpleDB is that this way the entire Dovecot system can be ephermeral. When a properly configured dovecot AMI is launched, it will start up, pull it's config data from an S3 bucket, subscribe to the SNS channel for new updates, and then start the Dovecot server. It won't care if it is the only Dovecot server, or if there are 500 other servers running. They all share the same simpleDB database. Whenever any change is made that is relevant to server configuration, a notice is generated to SNS, and all the email is stored in S3. As a starting point, I'm thinking the best place for me to start coding is the single-s3-dbox message store as it has the least moving parts[mainly just fix up the save function to run the way I need it to, and the retrieve function to make a local copy of any incoming email...additional metadata functionality can be added later]. Has anyone else been working on something similar? -Gary

4 10

[Dovecot] Mail migration to dovecot with doveadm backup
by Reinhard Vicinus 28 Jun '12

28 Jun '12

Hi, i try to migrate mails from a non dovecot imap server to a dovecot imap server with doveadm backup as described there: http://wiki2.dovecot.org/Migration/Dsync i first tried (local-mailbox port 18143 is the non dovecot imap server): /usr/bin/doveadm -o imapc_user=user(a)example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mailbox_list_index=no -v -D backup -R -f -u user(a)example.org -m Sent imapc: and got the following error: dsync(user(a)example.org): Fatal: dsync backup: Looks like you're trying to run backup in wrong direction. Source is empty and destination is not. As the dovecot imap account is newly created and therefore empty it seams to try to backup from the dovecot imap server to the non dovecot imap server. So i tried instead: /usr/bin/doveadm -o imapc_user=user(a)example.org -o imapc_password=imappw -o imapc_host=local-mailbox -o imapc_features=rfc822.size -o imapc_port=18143 -o mailbox_list_index=no -v -D backup -f -u user(a)example.org -m Sent imapc: Sometimes (every other time?) i got the following segmentation fault: bt #0 0x00007f15e2c9ed74 in strcasecmp () from /lib/libc.so.6 #1 0x00007f15e327eaff in imapc_save_callback (reply=0x7fff56096a70, context=<value optimized out>) at imapc-save.c:168 #2 0x00007f15e32853fe in imapc_command_reply_free (conn=0x72f040) at imapc-connection.c:946 #3 imapc_connection_input_tagged (conn=0x72f040) at imapc-connection.c:1039 #4 0x00007f15e3285668 in imapc_connection_input_one (conn=0x72f040) at imapc-connection.c:1085 #5 imapc_connection_input_pending (conn=0x72f040) at imapc-connection.c:1407 #6 0x00007f15e3285922 in imapc_connection_input (conn=0x72f040) at imapc-connection.c:1100 #7 0x00007f15e2fe6176 in io_loop_call_io (io=0x792510) at ioloop.c:379 #8 0x00007f15e2fe71ff in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:213 #9 0x00007f15e2fe6118 in io_loop_run (ioloop=0x7529a0) at ioloop.c:398 #10 0x00007f15e3281e49 in imapc_client_run_pre (client=0x7333e0) at imapc-client.c:142 #11 imapc_client_run (client=0x7333e0) at imapc-client.c:161 #12 0x00007f15e3280f24 in imapc_storage_run (storage=0x732bd0) at imapc-storage.c:118 #13 0x00007f15e327f003 in imapc_save_append (_ctx=0x74dcb0) at imapc-save.c:232 #14 imapc_save_finish (_ctx=0x74dcb0) at imapc-save.c:255 #15 0x00007f15e1bf06a1 in quota_save_finish (ctx=0x74dcb0) at quota-storage.c:227 #16 0x00007f15e3292487 in mailbox_save_finish (_ctx=0x7f15e2d4ca40) at mail-storage.c:1669 #17 0x000000000042b736 in local_worker_save_msg_continue (worker=0x73c770) at dsync-worker-local.c:1681 #18 0x000000000042b98c in local_worker_msg_save (_worker=0x73c770, msg=0x7f15e38e4298, data=0x7fff56096db0, callback=0x4269f0 <msg_save_callback>, context=0x78a610) at dsync-worker-local.c:1739 #19 0x000000000042b0d9 in dsync_worker_msg_save (worker=0x73c770, msg=<value optimized out>, data=0x7fff56096db0, callback=0x4269f0 <msg_save_callback>, context=0x78a610) at dsync-worker.c:234 #20 0x0000000000426ac5 in msg_get_callback (result=<value optimized out>, data=0x7fff56096db0, context=0x78a610) at dsync-brain-msgs-new.c:79 #21 0x000000000042dca9 in local_worker_msg_get_next (worker=0x726f30, get=0x7fff56096e00) at dsync-worker-local.c:1844 #22 0x000000000042def8 in local_worker_msg_get (_worker=0x0, mailbox=<value optimized out>, uid=3805596224, callback=0x2670, context=0x58) at dsync-worker-local.c:1865 #23 0x000000000042ace6 in dsync_worker_msg_get (worker=0x726f30, mailbox=0x7f15e38e40f1, uid=1, callback=0x426a40 <msg_get_callback>, context=<value optimized out>) at dsync-worker.c:261 #24 0x000000000042689e in dsync_brain_msg_sync_add_new_msg (iter=0x7f15e38e41d8) at dsync-brain-msgs-new.c:181 #25 dsync_brain_mailbox_add_new_msgs (iter=0x7f15e38e41d8) at dsync-brain-msgs-new.c:216 #26 dsync_brain_msg_sync_add_new_msgs (iter=0x7f15e38e41d8) at dsync-brain-msgs-new.c:315 #27 0x0000000000426164 in dsync_brain_msg_sync_more (sync=0x7f15e38e4050) at dsync-brain-msgs.c:436 #28 0x0000000000424979 in dsync_brain_sync_msgs (brain=0x7351c0) at dsync-brain.c:736 #29 dsync_brain_sync (brain=0x7351c0) at dsync-brain.c:857 #30 0x0000000000425849 in dsync_brain_subs_list_finished (context=0x750fa0) at dsync-brain.c:169 #31 dsync_worker_subs_input (context=0x750fa0) at dsync-brain.c:222 #32 0x0000000000424cbd in dsync_brain_sync (brain=0x7351c0) at dsync-brain.c:842 #33 0x00000000004256bc in dsync_brain_mailbox_list_finished (context=0x743a60) at dsync-brain.c:98 #34 dsync_worker_mailbox_input (context=0x743a60) at dsync-brain.c:125 #35 0x0000000000424afe in dsync_brain_sync (brain=0x7351c0) at dsync-brain.c:833 #36 0x0000000000425568 in dsync_brain_sync_all (brain=0x7351c0) at dsync-brain.c:897 #37 0x0000000000422ad6 in cmd_dsync_start (ctx=0x706560, worker1=<value optimized out>, worker2=<value optimized out>) at doveadm-dsync.c:342 #38 0x0000000000422dfa in cmd_dsync_run (_ctx=0x706560, user=0x7209c0) at doveadm-dsync.c:387 #39 0x000000000040f888 in doveadm_mail_next_user (ctx=0x706560, input=<value optimized out>, error_r=0x7fff560973d0) at doveadm-mail.c:311 #40 0x0000000000410071 in doveadm_mail_cmd (cmd=0x704880, argc=7, argv=0x6fe418) at doveadm-mail.c:518 #41 0x0000000000410501 in doveadm_mail_try_run (cmd_name=0x6fe4e4 "backup", argc=1443460960, argv=0x4377c7) at doveadm-mail.c:577 #42 0x00000000004170d1 in main (argc=7, argv=0x6fe3e8) at doveadm.c:373 The other times it shows the following error message (It seams to connect both times to the non dovecot imap server): doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_lookup (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Skipping module doveadm_zlib_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so: undefined symbol: i_stream_create_deflate (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_list_backend (this is usually intentional, so just ignore this message) doveadm(user(a)example.org): Debug: auth input: user(a)example.org home=/mail/dovecot/example.org/user uid=1000 gid=1000 quota_rule=*:bytes=2000M:messages=0 doveadm(user(a)example.org): Debug: Added userdb setting: plugin/quota_rule=*:bytes=2000M:messages=0 doveadm(user(a)example.org): Debug: Effective uid=1000, gid=1000, home=/mail/dovecot/example.org/user doveadm(user(a)example.org): Debug: Quota root: name=User quota backend=dict args=:proxy::quota doveadm(user(a)example.org): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 doveadm(user(a)example.org): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 doveadm(user(a)example.org): Debug: Quota warning: bytes=1992294400 (95%) messages=0 reverse=no command=quota-warning 95 user(a)example.org doveadm(user(a)example.org): Debug: Quota warning: bytes=1677721600 (80%) messages=0 reverse=no command=quota-warning 80 user(a)example.org doveadm(user(a)example.org): Debug: dict quota: user=user(a)example.org, uri=proxy::quota, noenforcing=0 doveadm(user(a)example.org): Debug: fs: root=/mail/dovecot/example.org/user/mail, index=, control=, inbox=, alt= doveadm(user(a)example.org): Debug: Namespace : Using permissions from /mail/dovecot/example.org/user/mail: mode=0700 gid=-1 dsync(user(a)example.org): Debug: Effective uid=1000, gid=1000, home=/mail/dovecot/example.org/user dsync(user(a)example.org): Debug: Quota root: name=User quota backend=dict args=:proxy::quota dsync(user(a)example.org): Debug: Quota rule: root=User quota mailbox=* bytes=2097152000 messages=0 dsync(user(a)example.org): Debug: Quota rule: root=User quota mailbox=Trash bytes=+104857600 messages=0 dsync(user(a)example.org): Debug: Quota warning: bytes=1992294400 (95%) messages=0 reverse=no command=quota-warning 95 user(a)example.org dsync(user(a)example.org): Debug: Quota warning: bytes=1677721600 (80%) messages=0 reverse=no command=quota-warning 80 user(a)example.org dsync(user(a)example.org): Debug: dict quota: user=user(a)example.org, uri=proxy::quota, noenforcing=0 dsync(user(a)example.org): Debug: imapc: root=, index=, control=, inbox=, alt= dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Looking up IP address dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Connecting to 10.10.10.10:18143 dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Server capabilities: IMAP4 IMAP4rev1 AUTH=LOGIN ACL NAMESPACE CHILDREN SORT QUOTA THREAD=ORDEREDSUBJECT UNSELECT IDLE dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Authenticating as user(a)example.org dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Authenticated successfully dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Looking up IP address dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Connecting to 10.10.10.10:18143 dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Server capabilities: IMAP4 IMAP4rev1 AUTH=LOGIN ACL NAMESPACE CHILDREN SORT QUOTA THREAD=ORDEREDSUBJECT UNSELECT IDLE dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Authenticating as user(a)example.org dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Authenticated successfully dsync(user(a)example.org): Warning: Destination mailbox Sent has been modified, need to recreate it before we can continue syncing dsync(user(a)example.org): Warning: Mailbox changes caused a desync. You may want to run dsync again. dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Disconnected dsync(user(a)example.org): Debug: imapc(local-mailbox:18143): Disconnected i think the problem could be that the account name on the remote server and the local server is absolute identical and doveadm backup has therefore problems discerning between the two locations. But that's only a stab in the dark and any help is appreciated. Kind regards Reinhard

3 9

[Dovecot] Default for non-present LDAP attributes?
by Edgar Fuß 28 Jun '12

28 Jun '12

With 1.2, is there a syntax to, for LDAP lookups, use a given fixed replacement for a non-present LDAP attribute? E.g. something that would extend user_attrs = mailFileServer=mail=maildir:/import/mail/%$/%d to use maildir:/import/mail/foo/%d in case the mailFileServer attribute is not present?

2 3

[Dovecot] Maildir Seen Flags not heeded when dovecot-shared present
by J E Lyon 28 Jun '12

28 Jun '12

Hi, After many hours of searching (!) and lots of testing procmail scripts, I found the explanation I was looking for -- something you explained at http://www.dovecot.org/list/dovecot/2008-July/032551.html That explains it. Thing is, though, every time I've seen shared mailboxes -- really shared by multiple staff or not -- the preference is in fact for the shared behaviours to also "share" the Seen flags. Typically, someone doesn't want to read an email that someone else has already picked up and started dealing with or responded to. (They'll file it in due course, but the Seen flag is the first indicator that someone's opened and started to deal with it.) Hacking source code and branching and whatnot isn't easy or done lightly, but I wondered if anything else had come to light in recent years about this issue. I'd be as happy getting my procmail script to tell Dovecot to update the index based on the flag, but I'm pretty sure that's not possible :) Any ideas greatly appreciated, thanks. ~ James.

2 3

[Dovecot] last hope... public namespace and directory structure
by Daniel Fischer 28 Jun '12

28 Jun '12

hello, I would like to migrate to dovecot, but I have a problem with a public namespace declaration: # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-2-amd64 x86_64 Debian wheezy/sid And here are the relevant parts from the configuration: namespace { location = maildir:/var/mail/vhosts/%d/public prefix = Public. separator = . type = public } namespace inbox { prefix = separator = . subscriptions = yes type = private } I assume that all folders under ./public/ are public mailboxes. The public folder itself is not a maildir, but contains the team mailboxes i.e. ./public/.sales/ ./public/.service/ ./public/.purchase/ The file passwd for those 3 samples looks like this: sales@$DOMAIN::5000:5000::/var/mail/vhosts/$DOMAIN/public/.sales service@$DOMAIN::5000:5000::/var/mail/vhosts/$DOMAIN/public/.service purchase@$DOMAIN::5000:5000::/var/mail/vhosts/$DOMAIN/public/.purchase Note: All other users have mail_location /var/mail/vhosts/%d/%n Now a have the following problem: If I login in as user sales and create a folder foo and in there a folder bar. The directory structure is: ./public/. sales /.foo and /public/. sales /.foo.bar thats exactly what I´m expect. Now I logon as normal user. I can see the namespace Public with the sales mailbox but no subfolder foo or foo.bar. Now I create also the folders foo and in there bar, but the result is to me unexpected ;-): ./public/. sales ./public/. sales.foo ./public/. sales.foo.bar Looking forward to your comment, Daniel

2 2

[Dovecot] permissions on auth-userdb
by spamvoll＠googlemail.com 28 Jun '12

28 Jun '12

Hi.. im still trying to upgrade to 2.0. Im getting: dovecot: lda: Error: userdb lookup: connect(/var/run/dovecot/auth-userdb) failed: Permission denied (euid=10000(vmail) egid=10000(vmail) missing +r perm: /var/run/dovecot/auth-userdb, euid is not dir owner) the error is correct caus its owned by root. My Questions is who should own it ? Im not sure how that works, what process/user calls the auth-userdb ? The auth-userdb returns the args generated in master.conf, right ? i think comment out the user and group setting in master.conf will fix it but im not sure if that is the securest way. the mails come from postfix via dovecot-lda Hans master.conf service auth { # auth_socket_path points to this userdb socket by default. It's typically # used by dovecot-lda, doveadm, possibly imap process, etc. Its default # permissions make it readable only by root, but you may need to relax these # permissions. Users that have access to this socket are able to get a list # of all usernames and get results of everyone's userdb lookups. unix_listener auth-userdb { mode = 0600 #user = vmail #group = vmail } auth-ldap.conf.ext passdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext } userdb { driver = static args = uid=vmail gid=vmail home=/home/MAILBOXES/%u/ mail=/home/MAILBOXES/%u/mail }

5 8