May 2009 - dovecot - dovecot.org

[Dovecot] Virtual users, local delivery via LDA and LDAP troubles
by Konstantin Khomoutov 19 May '09

19 May '09

We're investigating the possibiliy to migrate our mail system from Sendmail+Cyrus to Sendmail+Dovecot. The system must use authentication against Windows AD (supposedly using LDAP) and must use virtual hosting. So far we managed to work around a bug reported in [1], and IMAP/POP3 authentication on LDAP works OK. LDAP auth is set up using binds (Cyrus and Ejabberd authenticate against the same LDAP server without problems). As we use virtual users, userdb is set to be "static" in a standard way: userdb static { args = uid=10513 gid=10513 home=/var/local/dovecot/%u } After verifying IMAP/POP3 authentication works, I've set up the Dovecot LDA to deliver mail for domain users. This exposed another problem which I don't understand: the delivery program tries to figure out whether the user exists (which is perfectly sensible), it talks to the "master" authentication process which seemingly uses passdb backend to search LDAP. But this fails with the message "passdb doesn't support lookups, can't verify user's existence". [2] suggests it's auth binds that prevent this scheme from functioning correctly, but we can't stop using auth binds as Windows AD doesn't store users' passwords in any way sensible for external consumption. This would also pose unnecessary security risk on the domain, as the account used for initial binding should have had rights to read passwords, and its credentials are placed in the Dovecot configuration file in clear text. I read about "allow_all_users" in [3], but our Sendmail doesn't check whether the target user exists and we don't want to implement this as it logically pertains to the program which actually manages users' mailboxes -- Dovecot in our case. Is there a way to solve the problem at hand within the specified constraints? 1. http://dovecot.org/pipermail/dovecot/2009-May/039540.html 2. http://www.mail-archive.com/dovecot@dovecot.org/msg09449.html 3. http://www.mail-archive.com/dovecot@dovecot.org/msg08848.html

2 1

[Dovecot] what is deliver(1.1.10) doing in /tmp
by Pawel Eljasz 19 May '09

19 May '09

dear all, first part of log snippet is to show a bit of contrast, to me it's peculiar situation it's about messages between local users, actually one user, when I use command line mailx or anther local account to send to A_local user - ok second part is this weir error, it's selinux which prevents deliver write to /tmp, I believe thsese messages are from A_local user cron jobs(to himself) why would dovecot act differently in these scenarios, why would it need /tmp one time and not need it another May 19 14:32:29 whale postfix/pickup[9588]: 725F08F95500: uid=0 from=<root> May 19 14:32:29 whale postfix/cleanup[10877]: 725F08F95500: message-id=<20090519133229.725F08F95500(a)whale.hindhead.local> May 19 14:32:29 whale postfix/qmgr[3083]: 725F08F95500: from=<root(a)hindhead.local>, size=445, nrcpt=1 (queue active) May 19 14:32:30 whale dovecot: auth(default): new auth connection: pid=10883 May 19 14:32:30 whale postfix/smtpd[10883]: connect from localhost.localdomain[127.0.0.1] May 19 14:32:30 whale postfix/smtpd[10883]: 1E96A8F95501: client=localhost.localdomain[127.0.0.1] May 19 14:32:30 whale postfix/cleanup[10877]: 1E96A8F95501: message-id=<20090519133229.725F08F95500(a)whale.hindhead.local> May 19 14:32:30 whale postfix/qmgr[3083]: 1E96A8F95501: from=<root(a)hindhead.local>, size=906, nrcpt=1 (queue active) May 19 14:32:30 whale postfix/smtpd[10883]: disconnect from localhost.localdomain[127.0.0.1] May 19 14:32:30 whale amavis[9369]: (09369-14) Passed CLEAN, <root(a)hindhead.local> -> <pawel(a)hindhead.local>, Message-ID: <20090519133229.725F08F95500(a)whale.hindhead.local>, mail_id: D3qbEK-TL0Hi, Hits: 0.028, size: 445, queued_as: 1E96A8F95501, 650 ms May 19 14:32:30 whale postfix/smtp[10880]: 725F08F95500: to=<pawel(a)hindhead.local>, relay=127.0.0.1[127.0.0.1]:10024, delay=0.67, delays=0.02/0.01/0/0.65, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 1E96A8F95501) May 19 14:32:30 whale postfix/qmgr[3083]: 725F08F95500: removed May 19 14:32:30 whale deliver(pawel): Loading modules from directory: /usr/lib64/dovecot/lda May 19 14:32:30 whale deliver(pawel): Module loaded: /usr/lib64/dovecot/lda/lib90_cmusieve_plugin.so May 19 14:32:30 whale deliver(pawel): maildir: data=/home/pawel/Maildir May 19 14:32:30 whale deliver(pawel): maildir++: root=/home/pawel/Maildir, index=, control=, inbox=/home/pawel/Maildir _May 19 14:32:30 whale deliver(pawel): cmusieve: Using sieve path: /home/pawel/.dovecot.sieve May 19 14:32:30 whale deliver(pawel): cmusieve: Executing script /home/pawel/.dovecot.sievec_ May 19 14:32:30 whale deliver(pawel): msgid=<20090519133229.725F08F95500(a)whale.hindhead.local>: saved mail to INBOX May 19 14:32:30 whale postfix/local[10884]: 1E96A8F95501: to=<pawel(a)hindhead.local>, relay=local, delay=0.02, delays=0/0.01/0/0.01, dsn=2.0.0, status=sent (delivered to command: /usr/libexec/dovecot/deliver) May 19 14:32:30 whale postfix/qmgr[3083]: 1E96A8F95501: removed and right after above: May 19 14:33:05 whale postfix/pickup[9588]: 2607F8F95500: uid=501 from=<pawel> May 19 14:33:05 whale postfix/cleanup[10877]: 2607F8F95500: message-id=<20090519133305.2607F8F95500(a)whale.hindhead.local> May 19 14:33:05 whale postfix/qmgr[3083]: 2607F8F95500: from=<pawel(a)hindhead.local>, size=2892071, nrcpt=1 (queue active) May 19 14:33:05 whale postfix/smtpd[10883]: connect from localhost.localdomain[127.0.0.1] May 19 14:33:05 whale postfix/smtpd[10883]: 8DC458F95501: client=localhost.localdomain[127.0.0.1] May 19 14:33:05 whale postfix/cleanup[10877]: 8DC458F95501: message-id=<20090519133305.2607F8F95500(a)whale.hindhead.local> May 19 14:33:05 whale postfix/smtpd[10883]: disconnect from localhost.localdomain[127.0.0.1] May 19 14:33:05 whale postfix/qmgr[3083]: 8DC458F95501: from=<pawel(a)hindhead.local>, size=2899979, nrcpt=1 (queue active) May 19 14:33:05 whale deliver(pawel): Loading modules from directory: /usr/lib64/dovecot/lda May 19 14:33:05 whale deliver(pawel): Module loaded: /usr/lib64/dovecot/lda/lib90_cmusieve_plugin.so May 19 14:33:05 whale deliver(pawel): maildir: data=/home/pawel/Maildir May 19 14:33:05 whale deliver(pawel): maildir++: root=/home/pawel/Maildir, index=, control=, inbox=/home/pawel/Maildir May 19 14:33:05 whale deliver(pawel): maildir++: root=/home/pawel/Maildir, index=, control=, inbox=/home/pawel/Maildir _May 19 14:33:05 whale deliver(pawel): cmusieve: Using sieve path: /home/pawel/.dovecot.sieve May 19 14:33:05 whale deliver(pawel): cmusieve: Executing script /home/pawel/.dovecot.sievec_ *May 19 14:33:05 whale deliver(pawel): creat(/tmp/dovecot.deliver..1242739985.10903.8aa6fa47dffdc26d) failed: Permission denied May 19 14:33:05 whale deliver(pawel): copy: i_stream_read() failed: Permission denied* May 19 14:33:05 whale amavis[9395]: (09395-14) Passed CLEAN, <pawel(a)hindhead.local> -> <pawel(a)hindhead.local>, Message-ID: <20090519133305.2607F8F95500(a)whale.hindhead.local>, mail_id: QY0vutvX3AMN, Hits: -, size: 2896598, queued_as: 8DC458F95501, 449 ms May 19 14:33:05 whale postfix/smtp[10880]: 2607F8F95500: to=<pawel(a)hindhead.local>, orig_to=<pawel>, relay=127.0.0.1[127.0.0.1]:10024, delay=3, delays=2.5/0/0/0.45, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 8DC458F95501) May 19 14:33:05 whale deliver(pawel): msgid=<20090519133305.2607F8F95500(a)whale.hindhead.local>: save failed to INBOX: Internal error occurred. Refer to server log for more information. [2009-05-19 14:33:05] *May 19 14:33:05 whale deliver(pawel): sieve runtime error: Keep: Generic Error May 19 14:33:05 whale deliver(pawel): sieve_execute_bytecode(/home/pawel/.dovecot.sievec) failed* May 19 14:33:05 whale postfix/local[10884]: 8DC458F95501: to=<pawel(a)hindhead.local>, relay=local, delay=0.07, delays=0.06/0/0/0.01, dsn=4.3.0, status=deferred (temporary failure) May 19 14:33:05 whale postfix/qmgr[3083]: 2607F8F95500: removed cheers

2 1

[Dovecot] ext3/ext4 performance issue
by Timo Sirainen 19 May '09

19 May '09

Wonder if anyone else has actually noticed this and has some kind of statistics? http://marc.info/?t=124234003300001&r=1&w=2

6 8

[Dovecot] Segfault in deliver server
by Sascha Wilde 19 May '09

19 May '09

Hi Timo, Hi list, I finally got along to test the current dovecot 1.2 with our Kolab Server. And I'm very excited to see all the cool ACL and shared name spaces stuff upstream, thanks a lot Timo! Anyway I just stumbled across a new bug using our metadata-plugin (which in turn uses the dict back end): Making a few annotations requests after another it stops working. dovecot.log sais something like: Feb 03 11:58:21 burlywood3 <info> dovecot[10486]: IMAP(4(a)burlywood3.rgb): metadata_get_metadata_entry: dict key=shared//kolab/var/dovecot/spool/4@burlywood3. rgb/home/maildir/.Calendar//vendor/kolab/folder-type Feb 03 11:58:21 burlywood3 <error> dovecot[10486]: child 10503 (dict) killed with signal 11 Feb 03 11:58:21 burlywood3 <error> dovecot[10486]: IMAP(4(a)burlywood3.rgb): read(/kolab/var/dovecot/run/dict-server) failed: Connection reset by peer Feb 03 11:58:21 burlywood3 <critical> dovecot[10520]: Fatal: dup2(3) failed: Bad file descriptor Feb 03 11:58:21 burlywood3 <error> dovecot[10486]: child 10520 (dict) returned error 89 (Fatal failure) Feb 03 11:58:21 burlywood3 <critical> dovecot[10486]: Panic: file ioloop.c: line 38 (io_add): assertion failed: (fd >= 0) (END) I attached gdb to dict and got this: Program received signal SIGSEGV, Segmentation fault. array_idx_modifiable_i (array=0x38, idx=0) at array.c:10 10 pos = idx * array->element_size; (gdb) bt #0 array_idx_modifiable_i (array=0x38, idx=0) at array.c:10 #1 0x0805e9a2 in sql_pool_unlink (ctx=0x80fb670) at sql-pool.c:64 #2 0x0805ea24 in sql_pool_new (pool=0x80f9470, db_driver=0x80dd498 "sqlite", connect_string=0x810ad78 "/kolab/var/dovecot/lib/metadata-dict.sqlite") at sql-pool.c:97 #3 0x0805bb3c in sql_dict_init (driver=0x80f9ae0, uri=0xbfce9f76 "/kolab/etc/dovecot/metadata-dict.conf", value_type=DICT_DATA_TYPE_STRING, username=0x80fb910 "4(a)burlywood3.rgb") at dict-sql.c:86 #4 0x0805c9ca in dict_init (uri=0xbfce9f6f "sqlite:/kolab/etc/dovecot/metadata-dict.conf", value_type=DICT_DATA_TYPE_STRING, username=0x80fb910 "4(a)burlywood3.rgb") at dict.c:87 #5 0x0805a1b1 in dict_client_connection_input (conn=0x80fb8d0) at dict-server.c:407 #6 0x0806637c in io_loop_handler_run (ioloop=0x80f8a80) at ioloop-epoll.c:202 #7 0x080652fd in io_loop_run (ioloop=0x80f8a80) at ioloop.c:338 #8 0x0805a42d in main () at main.c:122 (gdb) li 5 6 void *array_idx_modifiable_i(struct array *array, unsigned int idx) 7 { 8 size_t pos; 9 10 pos = idx * array->element_size; 11 if (pos >= array->buffer->used) { 12 /* index doesn't exist yet, initialize with zero */ 13 buffer_append_zero(array->buffer, pos + array->element_size - 14 array->buffer->used); (gdb) p array $1 = (struct array *) 0x38 (gdb) p *array Cannot access memory at address 0x38 (gdb) up #1 0x0805e9a2 in sql_pool_unlink (ctx=0x80fb670) at sql-pool.c:64 64 next_ctx = SQL_POOL_CONTEXT(ctx->prev); (gdb) p *ctx $2 = {module_ctx = {reg = 0x0}, prev = 0x0, next = 0x810d2b0, pool = 0x80f9470, refcount = 0, key = 0x80fb638 "sqlite\t/kolab/var/dovecot/lib/metadata-dict.sqlite", orig_deinit = 0x805f229 <driver_sqlite_deinit_v>} (gdb) li 59 prev_ctx->next = ctx->next; 60 } 61 if (ctx->next == NULL) 62 ctx->pool->unused_head = ctx->prev; 63 else { 64 next_ctx = SQL_POOL_CONTEXT(ctx->prev); 65 next_ctx->prev = ctx->prev; 66 } 67 ctx->pool->unused_count--; 68 } (gdb) c Continuing. Program terminated with signal SIGSEGV, Segmentation fault. The program no longer exists It's quite possible, that we got any details in the usage of the dict back end wrong, but I guess that in any case the dict server shouldn't segfault... cheers sascha -- Sascha Wilde OpenPGP key: 4BB86568 http://www.intevation.de/~wilde/ http://www.intevation.de/ Intevation GmbH, Neuer Graben 17, 49074 Osnabrück; AG Osnabrück, HR B 18998 Geschäftsführer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner

2 2

[Dovecot] Multiple dovecot versions side-by-side
by Max Ivanov 19 May '09

19 May '09

I'm current maintainer of dovecot packages in ALTLinux distro and I plan to make possible to install multiple dovecot versions side-by-side (1.0, 1.1, 1.2) to make upgrade process easier and more robust. Is there any consequences of doing that? I worry about locking maildirs and mboxes, does dovecot handle it correctly if another dovecot process tries to work on same mail location?

2 1

[Dovecot] LDA and postfix with recipient delimiter: documentation
by John Fawcett 18 May '09

18 May '09

Just a comment about the documentation for postfix integration when using receipient delimiter and delivery of user+foo(a)domain.ext to user@domain: http://wiki.dovecot.org/LDA/Postfix I had some issues to get this working, so thought I'd share my experience. Dovecot version 1.2.rc3. Postfix version postfix-2.7-20090511. I have no local (unix) mailboxes, everything is delivered to dovecot, so in postfix I have set mailbox_transport = dovecot. In this case the following syntax taken from the docs does not work: dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${nexthop} whereas the following does work: dovecot unix - n n - - pipe flags=DRhu user=vmail:mail argv=/usr/lib/dovecot/deliver -f ${sender} -d ${user}@${domain} I think that ${domain} is always what is wanted and only when ${nexthop} happens to have the same value as ${domain} - for example when using a transport table entry like this: /etc/postfix/transport domain.ext dovecot: will the ${user}@${nexthop} form be deliverable. Can anyone see a reason for not using always ${user}@${domain} instead of ${user}@${nexthop}? John

5 6

[Dovecot] dovecot 2.0 (revision 9271:d467712aee77) compile problems on AIX 5.3
by Jonathan Siegle 18 May '09

18 May '09

Having some problems compiling on AIX 5.3 with IBM vac version 8. Programs that I had problems building: test-mail test-imap test-index To fix the undefined symbols problem: ld: 0711-317 ERROR: Undefined symbol: .charset_to_utf8_end ld: 0711-317 ERROR: Undefined symbol: .charset_to_utf8_begin ld: 0711-317 ERROR: Undefined symbol: .charset_to_utf8 ld: 0711-317 ERROR: Undefined symbol: .charset_is_utf8 ld: 0711-317 ERROR: Undefined symbol: .charset_to_utf8_str ld: 0711-317 ERROR: Undefined symbol: .message_header_decode_utf8 ld: 0711-317 ERROR: Undefined symbol: .rfc822_parser_init ld: 0711-317 ERROR: Undefined symbol: .rfc822_skip_lwsp ld: 0711-317 ERROR: Undefined symbol: .rfc822_parse_content_type ld: 0711-317 ERROR: Undefined symbol: .rfc2231_parse ld: 0711-317 ERROR: Undefined symbol: .rfc822_parse_mime_token ld: 0711-317 ERROR: Undefined symbol: .rfc822_parse_atom ld: 0711-317 ERROR: Undefined symbol: .message_address_parse ld: 0711-317 ERROR: Undefined symbol: .iconv ld: 0711-317 ERROR: Undefined symbol: .iconv_open ld: 0711-317 ERROR: Undefined symbol: .iconv_close I add this ../lib-charset/.libs/libcharset.a -liconv to the Makefiles (./src/lib-imap/Makefile;./src/lib-index/Makefile;./src/lib-mail/ Makefile) below: $ find . -name Makefile -exec egrep -p libcharset.a {} \; -print clean-noinstPROGRAMS: @list='$(noinst_PROGRAMS)'; for p in $$list; do \ f=`echo $$p|sed 's/$(EXEEXT)$$//'`; \ echo " rm -f $$p $$f"; \ rm -f $$p $$f ; \ done test-imap$(EXEEXT): $(test_imap_OBJECTS) $(test_imap_DEPENDENCIES) @rm -f test-imap$(EXEEXT) $(LINK) $(test_imap_LDFLAGS) $(test_imap_OBJECTS) $ (test_imap_LDADD) $(LIBS) \ ../lib-mail/.libs/libmail.a ../lib-charset/.libs/libcharset.a -liconv ./src/lib-imap/Makefile clean-noinstPROGRAMS: @list='$(noinst_PROGRAMS)'; for p in $$list; do \ f=`echo $$p|sed 's/$(EXEEXT)$$//'`; \ echo " rm -f $$p $$f"; \ rm -f $$p $$f ; \ done test-index$(EXEEXT): $(test_index_OBJECTS) $(test_index_DEPENDENCIES) @rm -f test-index$(EXEEXT) $(LINK) $(test_index_LDFLAGS) $(test_index_OBJECTS) $ (test_index_LDADD) $(LIBS) ../lib-charset/.libs/libcharset.a -liconv ./src/lib-index/Makefile clean-noinstPROGRAMS: @list='$(noinst_PROGRAMS)'; for p in $$list; do \ f=`echo $$p|sed 's/$(EXEEXT)$$//'`; \ echo " rm -f $$p $$f"; \ rm -f $$p $$f ; \ done test-mail$(EXEEXT): $(test_mail_OBJECTS) $(test_mail_DEPENDENCIES) @rm -f test-mail$(EXEEXT) $(LINK) $(test_mail_LDFLAGS) $(test_mail_OBJECTS) $ (test_mail_LDADD) $(LIBS) ../lib-charset/.libs/libcharset.a -liconv ./src/lib-mail/Makefile Then compilation occurs fine. Thanks! Jonathan

2 4

[Dovecot] How to manage Seen flags via sieve?
by Max Ivanov 18 May '09

18 May '09

I use public namespace for group mailboxes. If someone sends email from this mailbox , MTA detects it and delivers copy of that message to mailbox/Sent folder, to keep history of all messages available for every group member. I'd like to set Seen flag on all messages that dovecots deliver puts into "Sent" folder. Is it posible somehow?

2 1

[Dovecot] 1.1.13 squat core dump
by David Halik 18 May '09

18 May '09

While using 1.1.13 and squat with local pine imap on Solaris 9 I had a random core dump today. I was doing a search in a maildir folder with about 2000 emails. Unfortunately, I haven't been able to reproduce it yet, but I wanted to send along the info anyways. Environment: # uname -a SunOS 5.9 Generic_122300-39 sun4u sparc SUNW,Sun-Blade-100 Backtrace: http://pastebin.com/f7aa8bb5 ~# dovecot -n # 1.1.13: /usr/local/etc/dovecot.conf # OS: SunOS 5.9 sun4u protocols: imap imaps pop3 pop3s ssl_disable: yes disable_plaintext_auth: no login_dir: /usr/local/var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(pop3): /usr/local/libexec/dovecot/pop3-login login_greeting_capability(default): yes login_greeting_capability(imap): yes login_greeting_capability(pop3): no mail_location: maildir:~/Maildir:INDEX=/heroes/u1/dovecot-index/sparc/index/%u:CONTROL=/heroes/u1/dovecot-index/control/%u mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(pop3): /usr/local/libexec/dovecot/pop3 mail_plugins(default): quota imap_quota fts fts_squat mail_plugins(imap): quota imap_quota fts fts_squat mail_plugins(pop3): quota mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(pop3): /usr/local/lib/dovecot/pop3 pop3_uidl_format(default): %08Xu%08Xv pop3_uidl_format(imap): %08Xu%08Xv pop3_uidl_format(pop3): UID%u-%v namespace: type: private separator: . prefix: INBOX. inbox: yes list: yes subscriptions: yes auth default: passdb: driver: pam args: * userdb: driver: passwd plugin: quota: fs fts: squat fts_squat: partial=4 full=4 -- ================================ David Halik System Administrator OIT-CSS Rutgers University dhalik(a)jla.rutgers.edu ================================

5 14

Re: [Dovecot] LDA and postfix with recipient delimiter: documentation
by john.ml＠erba.tv 18 May '09

18 May '09

> Hi John, > no i didnt wrote that, i only have user(a)domain.ext > layout in use currently in my test servers > > -- > Best Regards > > MfG Robert Schetterer > > Germany/Munich/Bavaria > Hi Robert I probably didn't make it clear. The problem with the published syntax is that it does not work for receipient delimiter and delivery to mailbox, except as far as I can see when next_hop is the same as domain, which is not always the case. I think that the documentation should report the syntax with ${domain} instead of ${next_hop}, unless there's something I'm missing, since this should always work. thanks again John

2 1