March 2014 - dovecot - dovecot.org

[Dovecot] Finding memory leaks
by Philipp Brüll 02 May '14

02 May '14

Hello, I try to find a memory leak in a dovecot plugin that I develop. In order to find it, it would be helpful to print the total amount of memory that is currently allocated. This print could than spread over the source code and the memory consumption can be tracked. I've tried i_debug("pool size %u", (unsigned int)pool_alloconly_get_total_alloc_size(system_pool)); But that failed. Does someone has a good advice? I would like to avoid complex solutions like valgrind. Best regards, Philipp

2 1

[Dovecot] Imap user via proxy with starttls - how to rawlogs on proxy ?
by askforstatus 02 May '14

02 May '14

Hello, Is there any way to log user imap dialog with starttls when going via dovecot acting as imap proxy ? My intention is to log each and every user command similar to rawlogs feature. As per my current imap proxy config pre-login working like a charm but post-login is not possible due as i think the nature of proxy feature. Any idea ? Regards -- View this message in context: http://dovecot.2317879.n4.nabble.com/Imap-user-via-proxy-with-starttls-how-… Sent from the Dovecot mailing list archive at Nabble.com.

2 1

[Dovecot] Not backing up cache files
by Jesus Cea 02 May '14

02 May '14

I am using mdbox for dovecot storage. I wonder if I could skip "dovecot.index.cache" files and, in general, "*.cache" files when doing mail backup. Those files are big and change frequently. What I feel from reading documentation [1][2] is that they could be dropped in the backups, to be recreated on the fly if needed. But I would like expert confirmation. [1] http://wiki2.dovecot.org/IndexFiles [2] http://wiki2.dovecot.org/Design/Indexes/Cache Thanks!. -- Jesús Cea Avión _/_/ _/_/_/ _/_/_/ jcea(a)jcea.es - http://www.jcea.es/ _/_/ _/_/ _/_/ _/_/ _/_/ Twitter: @jcea _/_/ _/_/ _/_/_/_/_/ jabber / xmpp:jcea@jabber.org _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz

5 5

[Dovecot] Dovecot Replication setup
by Nikolaos Milas 18 Apr '14

18 Apr '14

Hi, I am trying to setup a master/master replication with dsync (according to the first part of: http://wiki2.dovecot.org/Replication) Sorry if this question sounds foolish, but I would like to ask/confirm whether the indicated setup is required on *both* servers participating in the master/master replication. It seems to me that both need it, but I would like to confirm, just in case. Both servers are running 2.2.12 on CentOS x86_64 (5.10 and 6.5 respectively) and users are virtual over ldap. I might have more questions in the near future, but this is a starting point. Please clarify. Best regards, Nick

3 5

[Dovecot] dsync replication questions
by Nikolaos Milas 07 Apr '14

07 Apr '14

I am running two servers with Dovecot v2.2.12 on CentOS x86_64 (5.10 and 6.5 respectively) and users are virtual over ldap. I have setup our main internal server (vmail.example.com) with dsync replication according to the first part of http://wiki2.dovecot.org/Replication. The second one (vmail1.example.com) will be the failover server which we want to be a real-time mirror (but can be manipulated directly and it should propagate changes back to vmail.example.com) When I started the server (vmail.example.com) mirroring started and completed fine (after a few hours). However, since then, I am not seeing a continued mirroring between the two. I would expect changes to one of the masters to be propagated in real time to the other - which does not happen. If I manually run (on vmail.example.com) dsync -u imaptester mirror ssh -l root vmail1.example.com dsync -u imaptester then these two accounts are synced. Question 1: Shouldn't mirroring continue automatically in real time (having configured it as in the first part of http://wiki2.dovecot.org/Replication)? If not, what should I do to enable such a real-time (or near-real-time) sync? Note that I have not configured "replication_dsync_parameters" as noted at the above page. Question 2: Where can I find documentation about "replication_dsync_parameters" and about "the doveadm sync command" (referred to at the same page)? I have not been able to locate such a documentation/wiki page. Question 2a: What do options "-d -N -l 30 -U" signify in "replication_dsync_parameters = -d -N -l 30 -U"? Thanks, Nick

4 11

[Dovecot] Crash in pop3 with version 2.2.12
by Urban Loesch 04 Apr '14

04 Apr '14

Hi, today I upgraded to version 2.2.12 under Debian Squeeze. I saw some people on the list they had the same problems with version 2.2.11, but which should have been fixed in version 2.2.12. My logfile shows: .. Mar 28 08:25:01 mailstore dovecot: pop3-login: Login: user=<ul(a)domain.net>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=34568, secured, session=<Jp6RmaX1YAB/AAAB> Mar 28 08:25:06 mailstore dovecot: pop3(ul(a)domain.net pid:34568 session:<Jp6RmaX1YAB/AAAB>): Fatal: master: service(pop3): child 34568 killed with signal 11 (core dumped) ... I made some more dubugging and I found that the pop3 process crashed on the "uidl" command. Here my output from telnet session: .. root@mailstore: # telnet localhost 110 Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. +OK Dovecot ready. user ul(a)domain.net +OK pass PASS +OK Logged in. list +OK 5 messages: 1 3492 2 21924 3 3525 4 3472 5 3273 . uidl Connection closed by foreign host. ... Listing massages or retrieving messages works normally. Only on "uidl" command the services crashes. I made a backtrace. -----> start backtrace <----- Core was generated by `dovecot/pop3'. Program terminated with signal 11, Segmentation fault. #0 0x00007f9dd8ca488d in vfprintf () from /lib/libc.so.6 (gdb) bt full #0 0x00007f9dd8ca488d in vfprintf () from /lib/libc.so.6 No symbol table info available. #1 0x00007f9dd8cc6732 in vsnprintf () from /lib/libc.so.6 No symbol table info available. #2 0x00007f9dd904d0db in str_vprintfa (str=0x11aa4f8, fmt=0x409184 "%u %s", args=0x7ffffab4fff0) at str.c:155 args2 = {{gp_offset = 16, fp_offset = 48, overflow_arg_area = 0x7ffffab500e0, reg_save_area = 0x7ffffab50010}} init_size = 4231558 pos = 0 ret = <value optimized out> ret2 = <value optimized out> __FUNCTION__ = "str_vprintfa" #3 0x00000000004055ff in client_send_line (client=0x11d1e50, fmt=<value optimized out>) at pop3-client.c:678 str = 0x11aa4f8 _data_stack_cur_id = 4 va = {{gp_offset = 32, fp_offset = 48, overflow_arg_area = 0x7ffffab500e0, reg_save_area = 0x7ffffab50010}} ret = <value optimized out> __FUNCTION__ = "client_send_line" #4 0x00000000004073cd in list_uidls_saved_iter (client=0x11d1e50, ctx=0x11da3c0) at pop3-commands.c:666 found = true #5 list_uids_iter (client=0x11d1e50, ctx=0x11da3c0) at pop3-commands.c:693 str = <value optimized out> permanent_uidl = <value optimized out> found = <value optimized out> failed = <value optimized out> #6 0x0000000000407d88 in cmd_uidl (client=0x11d1e50, name=<value optimized out>, args=0x408880 "") at pop3-commands.c:874 ctx = 0x0 seq = <value optimized out> #7 client_command_execute (client=0x11d1e50, name=<value optimized out>, args=0x408880 "") at pop3-commands.c:938 No locals. #8 0x0000000000405870 in client_handle_input (client=0x11d1e50) at pop3-client.c:739 _data_stack_cur_id = 3 line = <value optimized out> args = 0x408880 "" ret = <value optimized out> #9 0x00007f9dd903c3d6 in io_loop_call_io (io=0x11d2760) at ioloop.c:388 ioloop = 0x11b2740 t_id = 2 #10 0x00007f9dd903d45f in io_loop_handler_run (ioloop=<value optimized out>) at ioloop-epoll.c:220 ctx = 0x11b2aa0 event = 0x11b3900 list = 0x11b44d0 io = 0x51 tv = {tv_sec = 9, tv_usec = 999326} msecs = <value optimized out> ret = 1 i = 0 call = false __FUNCTION__ = "io_loop_handler_run" #11 0x00007f9dd903c348 in io_loop_run (ioloop=0x11b2740) at ioloop.c:412 __FUNCTION__ = "io_loop_run" #12 0x00007f9dd8fea953 in master_service_run (service=0x11b25d0, callback=0x409186) at master-service.c:566 No locals. #13 0x0000000000404ac7 in main (argc=1, argv=0x11b2390) at main.c:277 set_roots = {0x4094e0, 0x0} login_set = {auth_socket_path = 0x11aa050 "/var/run/dovecot/auth-master", postlogin_socket_path = 0x0, postlogin_timeout_secs = 60, callback = 0x404ca0 <login_client_connected>, failure_callback = 0x404c50 <login_client_failed>, request_auth_token = 0} service_flags = <value optimized out> storage_service_flags = MAIL_STORAGE_SERVICE_FLAG_DISALLOW_ROOT username = 0x0 c = <value optimized out> (gdb) quit -----> end backtrace <----- And at least my pop3 configuration part: -----> start config <----- protocol pop3 { mail_plugins = " quota mail_log notify acl zlib stats" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_lock_session = yes pop3_logout_format = bytes_sent=%o top=%t/%p, retr=%r/%b, del=%d/%m, size=%s uidl_hash=%u session=<%{session}> pop3_reuse_xuidl = yes } -----> end config <----- Setting "pop3_reuse_xuidl = no" has noe effect. Always the same crash. Thanks and regards Urban Loesch

4 6

[Dovecot] nfs flush/fsync config settings problem
by Troton_admin 03 Apr '14

03 Apr '14

Hi, I need to place a maildir storage on a NFS filesystem, so I set up the config like this: mmap_disable = yes mail_nfs_storage = yes mail_nfs_index = yes mail_fsync = always lock_method = dotlock (also tried fcntl - local lockd is running, no success, now using dotlock, to be sure while debugging) The problem is that Dovecot (2.2.9/2.2.10 - the relevant code in mail-index.c is the same) logs a fatal error and quits the session (no daemon crash/coredump). Jan 07 14:13:31 auth: Debug: client in: AUTH 1 PLAIN service=pop3 session=fRCRJGHvbADDUKU9 lip=192.168.100.102 rip=IP_ADDR lport=1110 rport=48236 resp=AHRyb3RvbjEAdHJvdG9uMQ== (previous base64 data may contain sensitive data) Jan 07 14:13:31 auth: Debug: passwd-file(troton1,IP_ADDR,<fRCRJGHvbADDUKU9>): lookup: user=troton1 file=/usr/local/etc/dovecot/passwd-file Jan 07 14:13:31 auth: Debug: client passdb out: OK 1 user=troton1 Jan 07 14:13:31 auth: Debug: master in: REQUEST 1289355265 29049 1 4f9fdd9e4069fa503e8e6bcc09d4cf30 session_pid=29089 Jan 07 14:13:31 auth: Debug: passwd-file(troton1,IP_ADDR,<fRCRJGHvbADDUKU9>): lookup: user=troton1 file=/usr/local/etc/dovecot/passwd-file Jan 07 14:13:31 auth: Debug: master userdb out: USER 1289355265 troton1 uid=81 gid=81 home=/home/troton1 mail=maildir:~/Maildir Jan 07 14:13:31 pop3-login: Info: Login: user=<troton1>, method=PLAIN, rip=IP_ADDR, lip=192.168.100.102, mpid=29089, session=<fRCRJGHvbADDUKU9> Jan 07 14:13:31 pop3: Debug: Added userdb setting: mail=maildir:~/Maildir Jan 07 14:13:31 pop3(troton1): Debug: Effective uid=81, gid=81, home=/home/troton1 Jan 07 14:13:31 pop3(troton1): Debug: Namespace inbox: type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=maildir:~/Maildir Jan 07 14:13:31 pop3(troton1): Debug: maildir++: root=/home/troton1/Maildir, index=, indexpvt=, control=, inbox=/home/troton1/Maildir, alt= Jan 07 14:13:31 pop3(troton1): Info: DEBUG1: fsync_mode=2 , mask=0 Jan 07 14:13:31 pop3(troton1): Info: DEBUG2: fsync_mode=2 flags=85 Jan 07 14:13:31 pop3(troton1): Debug: Namespace : Using permissions from /home/troton1/Maildir: mode=0700 gid=default Jan 07 14:13:31 pop3(troton1): Info: DEBUG2: fsync_mode=0 flags=85 The user/passdb is just a plaintext file, for debugging purposes. But what is strange, the messages system log shows: Jan 7 14:13:31 mailhub2 dovecot: pop3(troton1): Fatal: nfs flush requires mail_fsync=always That is obviously wrong, even its own doveconf -n utility shows the correct value. # doveconf -n | grep -E 'sync|nfs' mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes maildir_very_dirty_syncs = yes Affected code seems to be in the lib-index/mail-index.c file, mail_index_file function - around line 585 : // my custom debug line i_info("DEBUG2: fsync_mode=%u flags=%u", (unsigned int)(index->fsync_mode),(unsigned int)flags); if ((flags & MAIL_INDEX_OPEN_FLAG_NFS_FLUSH) != 0 && index->fsync_mode != FSYNC_MODE_ALWAYS) i_fatal("nfs flush requires mail_fsync=always"); The first time it sees fsync_mode=2 (which is correct value of FSYNC_MODE_ALWAYS), but next function call is with index->fsync_mode=0 and an error is raised, see the logfile above. Dovecot correctly creates its working files in user's home directory - dovecot-uidlist, dovecot-uidvalidity, dovecot-uidvalidity.52cbfd7b, dovecot.index.log . The complete (testing, not production) doveconf -n is: == # 2.2.10: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 7.3-RELEASE-p3 amd64 auth_debug = yes auth_debug_passwords = yes auth_mechanisms = plain login auth_verbose = yes auth_verbose_passwords = sha1 default_process_limit = 200 disable_plaintext_auth = no first_valid_uid = 81 imap_client_workarounds = delay-newmail tb-extra-mailbox-sep info_log_path = /var/log/dovecot.log last_valid_uid = 81 listen = 192.168.100.102 lock_method = dotlock mail_debug = yes mail_fsync = always mail_gid = 81 mail_location = maildir:%h/Maildir mail_nfs_index = yes mail_nfs_storage = yes mail_prefetch_count = 5 mail_uid = 81 mailbox_list_index = yes maildir_stat_dirs = yes maildir_very_dirty_syncs = yes mmap_disable = yes namespace inbox { inbox = yes location = mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . type = private } passdb { args = scheme=plain username_format=%n /usr/local/etc/dovecot/passwd-file driver = passwd-file } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid subject size } pop3_enable_last = yes pop3_save_uidl = yes protocols = imap pop3 service doveadm { inet_listener { port = 9292 } } service imap-login { inet_listener imap { port = 1143 } } service imap { process_limit = 250 } service pop3-login { inet_listener pop3 { port = 1110 } } service pop3 { process_limit = 200 } ssl = no userdb { args = username_format=%n /usr/local/etc/dovecot/passwd-file default_fields = uid=81 gid=0 home=/home/%n driver = passwd-file } protocol imap { mail_max_userip_connections = 5 } protocol pop3 { mail_max_userip_connections = 5 } == OS: 7.3-RELEASE-p3 FreeBSD 7.3-RELEASE-p3 #1 amd64 Affected Dovecot version: 2.2.9, 2.2.10 How to repeat: Anytime I enter the correct username/pass pair And I tried to place the homedir on a local disk too, and I got the same error as on NFS. Any ideas, what could possibly be wrong ? Is it a feature or a bug ? Thanks for your help. Regards, Troton

3 2

[Dovecot] Is replication broken?
by IT geek 31 03 Apr '14

03 Apr '14

Hi, Apologies for this being my 248th post about replication... The current error I'm getting is: Jan 30 12:10:54 Server1 dovecot: dsync-local(MailUser1): Error: dsync( Server2.test.com) read() failed: read((fd)) failed: Connection reset by peer Subsequent errors are: Jan 30 12:15:47 Server1 dovecot: dsync-local(MailUser1): Error: read( Server2.test.com) failed: EOF The above error is seen on Server1, after the initial sync completes of the first mailbox (Sent folder to be precise). Does anyone have an ideas as to what might be causing it? These are brand new rebuilds, running the latest build - I just can't seem to get this working. Bug maybe? Configs are as follows: Server1: # 2.2.10: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 5.2 cobalt auth_mechanisms = plain login doveadm_password = secret doveadm_port = 12345 first_valid_uid = 1003 last_valid_uid = 1020 listen = 192.168.1.1 login_greeting = test.com mail_location = mbox:~/mail:INBOX=/var/mail/%u mail_plugins = " notify replication" passdb { driver = passwd } plugin { mail_replica = tcp:Server2.test.com } protocols = imap service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0600 } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service imap-login { inet_listener imap { port = 0 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl_cert = </etc/openssl/certs/Server1.test.crt ssl_key = </etc/openssl/private/Server1.test.key userdb { driver = passwd } protocol lda { postmaster_address = postmaster(a)test.com } ServerB: # 2.2.10: /usr/pkg/etc/dovecot/dovecot.conf # OS: NetBSD 5.1 cobalt auth_mechanisms = plain login doveadm_password = secret doveadm_port = 12345 listen = 192.168.1.2 login_greeting = test.com mail_location = mbox:~/mail:INBOX=/var/mail/%u passdb { driver = passwd } protocols = imap service aggregator { fifo_listener replication-notify-fifo { mode = 0666 } unix_listener replication-notify { mode = 0600 } } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } user = root } service doveadm { inet_listener { port = 12345 } } service imap-login { inet_listener imap { port = 0 } } service replicator { process_min_avail = 1 unix_listener replicator-doveadm { mode = 0600 } } ssl_cert = </etc/openssl/certs/Server2.test.com.crt ssl_key = </etc/openssl/private/Server2.test.com.key userdb { driver = passwd } protocol lda { postmaster_address = postmaster(a)test.com }

4 15

[Dovecot] Dovecot not honoring configuration settings (auth failure)
by Jeffrey Walton 01 Apr '14

01 Apr '14

My dovecot.conf has the following: # No results when searching the wiki disable_plaintext_auth = no # http://wiki2.dovecot.org/Authentication/Mechanisms auth_mechanisms = plain login digest-md5 cram-md5 When I attempt to run imapsync, I receive an error: Host2: <host> says it has NO CAPABILITY for AUTHENTICATE LOGIN imapsync also dumps the helo string, and it is missing: Host2: * OK [CAPABILITY IMAP4rev1 LITERAL+ ... STARTTLS AUTH=PLAIN] Dovecot ready. I've restarted the dovecot service with 'service dovecot restart' and even rebooted the machine. There is nothing reported in any on the log files (/var/mail/dovecot.log and /var/log/mail.<level>). Any ideas why dovecot is not honoring the setting in its config file? ********** # dovecot --version 2.1.7 # doveconf -n # 2.1.7: /etc/dovecot/dovecot.conf # OS: Linux 3.2.0-4-686-pae i686 Debian 7.4 disable_plaintext_auth = no log_path = /var/log/dovecot.log mail_location = mbox:~/mail:INBOX=/var/mail/%u namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } ... prefix = } passdb { args = /var/mail/%d/users driver = passwd-file } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = " imap pop3" ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { driver = passwd }

5 12

[Dovecot] Deduplicate not processing all messages - bug?
by Jiri Bourek 01 Apr '14

01 Apr '14

Hello, I'm trying to create automated backup recovery using "doveadm import" and "doveadm deduplicate". During testing I noticed that deduplicate only deletes some duplicates and has to be called multiple times to find them all. Here's what I've been trying (in shell commands): First, expunge inbox (the end result is the same even if you delete only some messages): # doveadm expunge -u test mailbox inbox all # ls /home/mailboxes/test/cur | wc -l 0 Then import data from backup - twice, so duplicates are created (again, if you don't delete all messages and call import only once, resulting behaviour is the same.) # doveadm import -u test maildir:/home/test "" mailbox INBOX # doveadm import -u test maildir:/home/test "" mailbox INBOX # ls /home/mailboxes/test/cur | wc -l 1046 Then try to deduplicate # doveadm deduplicate -u test mailbox INBOX # ls /home/mailboxes/test/cur | wc -l 1040 And again # doveadm deduplicate -u test mailbox INBOX # ls /home/mailboxes/test/cur | wc -l 1029 And so on until the message count holds on 523 Each repetition removes 10 - 30 duplicates so eventually all duplicates are removed if "doveadm deduplicate" is called enough times in a row. I also noticed that when I repeat the test, import the backup again and call deduplicate, the steps - how many messages are removed at one time - are the same. That is I start with 1046 messages in the mailbox, after first run there's 1040, then 1029 and so on. My guess would be the behaviour depends on what is stored in the mailbox, but that's pretty much all I can figure out on my own at this time. My question is - is this intended behaviour, ie. are you supposed to run doveadm deduplicate as long as the number of messages in the mailbox keeps changing? Or is it a bug? Tried to Google for the answer but no luck, so thanks for any answers. Tested on Dovecot version 2.2.9 and 2.2.12 (both from Debian repositories.)

2 1