January 2011 - dovecot

[Dovecot] squat-trie.c: line 876 (squat_build_word): assertion failed: (i + bytelen <= size)
by Ralf Hildebrandt 07 Jan '11

07 Jan '11

>From the log: Jan 7 16:16:44 postamt dovecot: imap(hxxxxnge): Panic: file squat-trie.c: line 876 (squat_build_word): assertion failed: (i + bytelen <= size) Jan 7 16:16:44 postamt dovecot: imap(hxxxxnge): Error: Raw backtrace: /usr/dovecot-2/lib/dovecot/libdovecot.so.0(+0x3bbe1) [0xb7745be1] -> /usr/dovecot-2/lib/dovecot/libdovecot.so.0(+0x3bc4f) [0xb7745c4f] -> /usr/dovecot-2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb771c82e] -> /usr/dovecot-2/lib/dovecot/lib21_fts_squat_plugin.so(+0x566a) [0xb752a66a] -> /usr/dovecot-2/lib/dovecot/lib21_fts_squat_plugin.so(squat_trie_build_more+0x126)[0xb752aee6] -> /usr/dovecot-2/lib/dovecot/lib21_fts_squat_plugin.so(+0x2b9c) [0xb7527b9c] -> /usr/dovecot-2/lib/dovecot/lib20_fts_plugin.so(fts_backend_build_more+0x1f)[0xb75735af] -> /usr/dovecot-2/lib/dovecot/lib20_fts_plugin.so(+0x578c) [0xb757678c] -> /usr/dovecot-2/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x38) [0xb77a5a78] -> dovecot/imap [hxxxxnge 141.42.206.38 UID](+0x168d6) [0xb78788d6] -> dovecot/imap [hxxxxnge 141.42.206.38 UID](imap_search_start+0x138) [0xb78792d8] -> dovecot/imap [hxxxxnge 141.42.206.38 UID](cmd_search+0xf1) [0xb786e4d1] -> d Jan 7 16:16:44 postamt dovecot: master: Error: service(imap): child 16959 killed with signal 6 (core dumped) Jan 7 16:16:50 postamt dovecot: imap(hxxxxnge): Panic: file squat-trie.c: line 876 (squat_build_word): assertion failed: (i + bytelen <= size) Jan 7 16:16:50 postamt dovecot: imap(hxxxxnge): Error: Raw backtrace: /usr/dovecot-2/lib/dovecot/libdovecot.so.0(+0x3bbe1) [0xb7738be1] -> /usr/dovecot-2/lib/dovecot/libdovecot.so.0(+0x3bc4f) [0xb7738c4f] -> /usr/dovecot-2/lib/dovecot/libdovecot.so.0(i_fatal+0) [0xb770f82e] -> /usr/dovecot-2/lib/dovecot/lib21_fts_squat_plugin.so(+0x566a) [0xb751d66a] -> /usr/dovecot-2/lib/dovecot/lib21_fts_squat_plugin.so(squat_trie_build_more+0x126)[0xb751dee6] -> /usr/dovecot-2/lib/dovecot/lib21_fts_squat_plugin.so(+0x2b9c) [0xb751ab9c] -> /usr/dovecot-2/lib/dovecot/lib20_fts_plugin.so(fts_backend_build_more+0x1f)[0xb75665af] -> /usr/dovecot-2/lib/dovecot/lib20_fts_plugin.so(+0x578c) [0xb756978c] -> /usr/dovecot-2/lib/dovecot/libdovecot-storage.so.0(mailbox_search_next_nonblock+0x38) [0xb7798a78] -> dovecot/imap [hxxxxnge 141.42.206.38 UID](+0x168d6) [0xb786b8d6] -> dovecot/imap [hxxxxnge 141.42.206.38 UID](imap_search_start+0x138) [0xb786c2d8] -> dovecot/imap [hxxxxnge 141.42.206.38 UID](cmd_search+0xf1) [0xb78614d1] -> d Jan 7 16:16:50 postamt dovecot: master: Error: service(imap): child 17558 killed with signal 6 (core dumped) -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

1 0

[Dovecot] dovecot can't see mail that is there
by Joachim Ferreira 07 Jan '11

07 Jan '11

I used to install dovecot on any Fedora fresh installation and I could fetch my internal mail few minutes later. Now I've been struggling for two days and I haven't got my mail working yet. I have Fedora 14 with sendmail. Some application on the same machine send mail to an account on the same machine (hence all is confined to one machine) and I want to retrieve it from a laptop on the same LAN via Outlook. If I login into the account and check the mail with the alpine (ex pine) client, I see that there are 10 mails. When I "telnet localhost pop3" and login into the account, the "list" command says there are 0 mails. Various suggestions found on the internet for slightly different configuration did not help. All the help will be very much appreciated. Thanks, Joachim ++++++++++++++++++++++++++++++++++++++++++++ The output of dovecot -n ------------------------ # dovecot -n # 2.0.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.35.10-74.fc14.x86_64 x86_64 Fedora release 14 (Laughlin) auth_debug = yes auth_debug_passwords = yes auth_verbose_passwords = plain disable_plaintext_auth = no mail_location = mbox:INBOX=/var/mail/%u mbox_write_locks = fcntl passdb { driver = pam } pop3_client_workarounds = outlook-no-nuls oe-ns-eoh ssl_cert = </etc/pki/dovecot/certs/dovecot.pem ssl_key = </etc/pki/dovecot/private/dovecot.pem userdb { driver = passwd } NOTE - I don't use ssl, but disabling ssl + certificate did not change behaviour. THE OUTPUT OF /var/log/maillog ----------------------------------------- Jan 6 19:14:26 BEATLES dovecot: auth: Debug: Loading modules from directory: /usr/lib64/dovecot/auth Jan 6 19:14:26 BEATLES dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libauthdb_ldap.so Jan 6 19:14:26 BEATLES dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libdriver_sqlite.so Jan 6 19:14:26 BEATLES dovecot: auth: Debug: Module loaded: /usr/lib64/dovecot/auth/libmech_gssapi.so Jan 6 19:14:26 BEATLES dovecot: auth: Debug: pam(joachim,192.168.0.88): lookup service=dovecot Jan 6 19:14:26 BEATLES dovecot: auth: Debug: pam(joachim,192.168.0.88): #1/1 style=1 msg=Password: Jan 6 19:14:26 BEATLES dovecot: auth: Debug: client out: OK#0111#011user=joachim Jan 6 19:14:26 BEATLES dovecot: auth: Debug: master in: REQUEST#0113019243521#01112409#0111#0116f7f83729469c6158f08283ec2e20067 Jan 6 19:14:26 BEATLES dovecot: auth: Debug: passwd(joachim,192.168.0.88): lookup Jan 6 19:14:26 BEATLES dovecot: auth: Debug: master out: USER#0113019243521#011joachim#011system_groups_user=joachim#011uid=500#011gi d=500#011home=/home/joachim Jan 6 19:14:26 BEATLES dovecot: pop3-login: Login: user=<joachim>, method=PLAIN, rip=192.168.0.88, lip=192.168.0.2, mpid=12414 Jan 6 19:14:26 BEATLES dovecot: pop3(joachim): Disconnected: Logged out top=0/0, retr=0/0, del=0/0, size=0

2 2

[Dovecot] Mail delivered, but lost?
by Patrick Ben Koetter 07 Jan '11

07 Jan '11

Running dovecot 1.2.9 on a Ubuntu 10.04.1 LTS virtual machine. Postfix receives messages and hands them over to deliver for local storage. Today two mails came in that could not be found allthough they got delivered. What catches my eye is the "?" in the msgid: Jan 5 21:10:47 mail dovecot: deliver(consulting): msgid=? <14847_1294258245_ZZh0g1h9eLfxL.00_fc2fe6e910f03.4d24de54(a)uni-bielefeld.de>: saved mail to INBOX Jan 5 21:27:26 mail dovecot: deliver(consulting): msgid=? <18995_1294259246_ZZh0g1hhis~T6.00_fc45b33d1281c.4d24e23d(a)uni-bielefeld.de>: saved mail to INBOX Is that a known error? Anything I can do to fix that? p@rick -- state of mind Digitale Kommunikation http://www.state-of-mind.de Franziskanerstraße 15 Telefon +49 89 3090 4664 81669 München Telefax +49 89 3090 4666 Amtsgericht München Partnerschaftsregister PR 563

2 1

[Dovecot] dsync backup errors with maildir -> mdbox
by Patrick McLean 06 Jan '11

06 Jan '11

Hi, In preparation for a migration from maildir to mdbox, I am using dsync backup to do initial mailbox copies (at this point I do not want to risk any changes to the maildir). The initial run of dsync runs fine, but a second run to migrate any changes since the first run produces errors for many users. Here is an example of the type of error produced: > # dsync -u user backup mdbox:~/mdbox > # dsync -u user backup mdbox:~/mdbox > dsync(user): Error: Can't delete mailbox INBOX: INBOX can't be deleted. > dsync(user): Error: Can't delete mailbox directory XXXXXX.courses: Mailbox has children, delete them first > dsync(user): Error: Can't delete mailbox directory XXXXXX: Mailbox has children, delete them first > dsync(user): Error: Can't delete mailbox directory XXXXXX.XX1: Mailbox has children, delete them first > dsync(user): Error: Can't delete mailbox directory XXXXXX.XX2: Mailbox has children, delete them first > dsync(user): Error: Can't delete mailbox directory XXXXXXXX: Mailbox has children, delete them first > dsync(user): Error: Can't delete mailbox directory XXX: Mailbox has children, delete them first > dsync(user): Error: Trying to open a non-listed mailbox with guid=c0204a140fdd254d1a770000743562c1 > dsync(user): Error: msg iteration failed: Couldn't open mailbox c0204a140fdd254d1a770000743562c1 > dsync(user): Error: Trying to open a non-listed mailbox with guid=c0204a140fdd254d1a770000743562c1

1 0

[Dovecot] Alternative storage vs. Self healing (losing flags?)
by Marczisovszky Daniel 06 Jan '11

06 Jan '11

Hi, We are going to set up an environment for our customers where the storage is provided by two servers. First one stores the new emails, second one stores all the others using Dovecot's alternative storage feature (mailbox format is mdbox). The servers are connected via NFS. If, for any reason, the alternative storage is not mounted, Dovecot will see only an empty folder and due to its self-healing functionality flags for messages stored on alternative storage will be lost. Is there any way to prevent regenerating indexes (and especially flags as a side-effect) if the alternative storage seems to be completely empty? (like a dummy file in the root of the alternative storage and an option to enable/disable this check) Thanks in advance, Daniel Marczisovszky

1 0

[Dovecot] Login with firstname.lastname instead of loginname - WORKING
by Ralf Hildebrandt 06 Jan '11

06 Jan '11

3 years ago (!) I aksed about this and finally I implemented it today: >> # NEW >> passdb { >> driver = passwd >> # firstname.lastname@charite.de:password:uid:gid:(gecos):home:(shell):user=realloginname >> args = /usr/local/etc/dovecot.login_via_email >> } >If you're only using it as a passdb, you don't need the uid/gid/home. So >an example field would be: > >firstname.lastname@charite.de:password::::::user=realloginname But now I'm encountering problems with users writing Firstname.Lastname(a)charite.de instead of firstname.lastname(a)charite.de (capitalized first letters) How can I "normalize" the usernames by lowercasing them? The coolest solution would be to strip realms AND force lowercasing of usernames in one pass -- Ralf Hildebrandt Geschäftsbereich IT | Abteilung Netzwerk Charité - Universitätsmedizin Berlin Campus Benjamin Franklin Hindenburgdamm 30 | D-12203 Berlin Tel. +49 30 450 570 155 | Fax: +49 30 450 570 962 ralf.hildebrandt(a)charite.de | http://www.charite.de

4 5

[Dovecot] Upgrade from 1.2.16 with sieve/managesieve to 2.0.8 with pigeonhole
by Nick Rosier 06 Jan '11

06 Jan '11

Hi, I would like to upgrade my server from 1.2.16 with sieve and managesieve to 2.0.8 with pigeonhole. Will I get all functionality I was having before? Should this migration be painless or will it require some additional work? I've read the Upgrading guide so I know I need to create an additional user. I've migrated the mails from Maildir to dbox in the past (using the convert plugin); does that mean I'm fully dbox or am I in a maildir-dbox hybrid situation? I've noticed that there are still some maildir-formatted mail (e.g. 1259960330.M322229P62710.mail,W=2195:2,S) in some of the maps. root@mail /usr/local/etc # dovecot -n # 1.2.16: /usr/local/etc/dovecot.conf # OS: FreeBSD 8.2-PRERELEASE amd64 zfs protocols: imap imaps managesieve ssl_ca_file: /etc/ssl/cert.pem ssl_cert_file: /etc/ssl/mail.crt ssl_key_file: /etc/ssl/mail.key version_ignore: yes login_dir: /var/run/dovecot/login login_executable(default): /usr/local/libexec/dovecot/imap-login login_executable(imap): /usr/local/libexec/dovecot/imap-login login_executable(managesieve): /usr/local/libexec/dovecot/managesieve-login valid_chroot_dirs: /var/spool/vmail mail_max_userip_connections(default): 50 mail_max_userip_connections(imap): 50 mail_max_userip_connections(managesieve): 10 verbose_proctitle: yes first_valid_uid: 5000 first_valid_gid: 5000 mail_privileged_group: mail mail_location: dbox:/var/spool/vmail/%d/%n mail_executable(default): /usr/local/libexec/dovecot/imap mail_executable(imap): /usr/local/libexec/dovecot/imap mail_executable(managesieve): /usr/local/libexec/dovecot/managesieve mail_plugins(default): acl imap_acl fts fts_squat autocreate mail_plugins(imap): acl imap_acl fts fts_squat autocreate mail_plugins(managesieve): mail_plugin_dir(default): /usr/local/lib/dovecot/imap mail_plugin_dir(imap): /usr/local/lib/dovecot/imap mail_plugin_dir(managesieve): /usr/local/lib/dovecot/managesieve imap_client_workarounds(default): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(imap): delay-newmail netscape-eoh tb-extra-mailbox-sep imap_client_workarounds(managesieve): dict_db_config: /usr/local/etc/dovecot-db.conf namespace: type: private separator: / inbox: yes list: yes subscriptions: yes lda: postmaster_address: postmaster(a)xxx.xxx mail_plugins: sieve acl mail_plugin_dir: /usr/local/lib/dovecot/lda deliver_log_format: msgid=%m: %$ sendmail_path: /usr/sbin/sendmail auth default: mechanisms: plain login cram-md5 username_format: %Lu passdb: driver: sql args: /usr/local/etc/dovecot-sql.conf userdb: driver: sql args: /usr/local/etc/dovecot-sql.conf socket: type: listen client: path: /var/spool/postfix/private/auth mode: 432 user: postfix group: postfix master: path: /var/run/dovecot/auth-master mode: 384 user: vmail group: vmail plugin: acl: vfile:/usr/local/etc/dovecot/acls:cache_secs=300 acl_shared_dict: file:/var/db/dovecot/shared-mailboxes.db convert_mail: maildir:/var/spool/vmail/%d/%n expire_dict: proxy::expire expire: Trash 365 sieve: /var/spool/vmail/%d/%n/dovecot.sieve sieve_dir: /var/spool/vmail/%d/%n/sieve sieve_global_dir: /usr/local/etc/dovecot/sieve fts: squat fts_squat: partial=4 full=10 autocreate: Trash autocreate2: Drafts autocreate3: Spam autocreate4: Sent autosubscribe: Trash autosubscribe2: Drafts autosubscribe3: Spam autosubscribe4: Sent dict: expire: db:/var/db/dovecot/expire.db

1 0

[Dovecot] RFC: grouped (f)sync
by Attila Nagy 06 Jan '11

06 Jan '11

Hello, Currently Dovecot (and any other application, which cares about e-mail delivery) does at least one fsync per mail delivery. Given that hard disk drives have a very limited IOPS, this effectively limits the maximum mail delivery performance to a very low value, under utilizing the available storage IO capacity. Calculating with an average mail size of 50 kB and an average consumer HDD with 120 IOPS, the theoretical mail delivery performance will be 50 kB*120 IOPS=5.85 MBps. But if we could write 500 kB with every transaction, the delivery speed would be nearly 10 times as well. Dovecot have two process models: separate processes for each client connection and an async in-process multiplexing method. This works for each one, albeit the timing is somewhat different. So here's the idea: instead of fsyncing immediately in the LDA (lmtpd) every time when the client says "\r\n.\r\n" after the DATA phase, let's introduce a user settable timer (let's call that sync_delay from now on) and only sync in every sync_delay seconds. This would introduce an up to sync_delay seconds delay in lmtpd returning "250 Ok" to the client, but that's generally not a problem, because in high traffic setups there is a great amount of concurrency, so you could use a lot of client connections easily. Take an example setting of sync_delay = 100 ms. With this, 10 syncs would happen in every second from Dovecot LDA processes, meaning if a client connects in t=0 it will immediately got the response 250, if a client connects in t=0.05, it will get the response in 50 ms (in an ideal world, where syncing does not take time), and the committed blocks could accumulate for a maximum of 100 ms. In a busy system (where this setting would make sense), it means it would be possible to write more data with less IOPS needed. I can see two problems: 1. there is no call for committing a lot of file descriptors in one transaction, so instead of fsync() for each of the modified FDs, a sync() would be needed. sync() writes all buffers to stable storage, which is bad if you have a mixed workload, where there are a lot of non-fsynced data, or other heavy fsync users. But modern file systems, like ZFS will write those back too, so there an fsync(fd) is -AFAIK- mostly equivalent with a sync(pool on which fd is). sync() of course is system wide, so if you have other file systems, those will be synced as well. (this setting isn't for everybody) 2. in a multiprocess environment this would need coordination, so instead of doing fsyncs in distinct processes, there would be one process needed, which does the sync and returns OK for the others, so they can notify the client about the commit to the stable storage. Any opinions on this?

2 2

[Dovecot] virtual user problems
by G.Nau 06 Jan '11

06 Jan '11

dovecot 2.0.8 with postfix; transfer postfix -> dovecot by lmtp is fine Now i have to redirect a couple of virtual email names (like admin(a)mydomain.com, postmaster@, webmaster@, info@, sales@ ...) to a single user postbox named xxx@mydomain. As I use mysql for the auth-sql, it's returning userhome /home/<name>/maildir for all sql requests to <name>=admin, postmaster etc. I also verified that "doveadm user info" is returning the maildir/home: /home/xxx/maildir So I assumed, that all mails to admin, postmaster etc. are transfered to user xxx mailbox. But it isn't. Dovecot is creating the maildirs /home/admin/maildir, /home/info/maildir and so on. What do I miss here to get that virtual users mapped to the correct user xxx? Regards Birte

1 0

[Dovecot] good value of vsz_limit
by fakessh ＠ 06 Jan '11

06 Jan '11

hello dovecot network hello Timo I've just upgraded to 2.0.8 dovecot and I wonder it should be good value of vsz_limit I put value as 64 M but I do not know if it's good value return are welcome sincerely and happy new year this my dovecot -n ~]# /usr/sbin/dovecot -n # 2.0.8: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32.2-xxxx-grs-ipv6-32 i686 CentOS release 5.5 (Final) auth_mechanisms = plain login base_dir = /var/run/dovecot/ imap_capability = +IDLE listen = [::] log_path = /var/log/maillog log_timestamp = %Y-%m-%d %H:%M:%S login_log_format_elements = user=<%u> method=%m rip=%r lip=%l %c mail_location = maildir:~/Maildir mail_max_userip_connections = 20 managesieve_notify_capability = mailto managesieve_sieve_capability = comparator-i;octet comparator-i;ascii-casemap fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date spamtest spamtestplus virustest namespace { inbox = yes location = prefix = separator = . } passdb { driver = pam } plugin { plugin = autocreate managesieve sieve sieve = ~/.dovecot.sieve sieve_before = /var/sieve-scripts/roundcube.sieve sieve_dir = ~/sieve sieve_global_path = whatever } protocols = sieve imap pop3 service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-master { mode = 0666 } vsz_limit = 64 M } service imap-login { inet_listener imap { port = 0 } inet_listener imaps { address = * , [::] port = 993 } process_limit = 128 vsz_limit = 64 M } service imap { client_limit = 5 service_count = 0 } service managesieve-login { inet_listener managesieve-login { address = * port = 2000 } process_limit = 128 vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 0 } inet_listener pop3s { address = * , [::] port = 995 } process_limit = 128 vsz_limit = 64 M } ssl_ca = </etc/pki/tls/certs/root.crt ssl_cert = </etc/pki/tls/certs/r13151.ovh.net.crt ssl_key = </etc/pki/tls/private/r13151.ovh.net.key ssl_verify_client_cert = yes userdb { driver = passwd } version_ignore = yes protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep imap_max_line_length = 64 k mail_plugins = autocreate } protocol pop3 { mail_plugins = autocreate pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_logout_format = top=%t/%p, retr=%r/%b, del=%d/%m, size=%s } protocol lda { hostname = r13151.ovh.net mail_plugins = autocreate sieve postmaster_address = postmaster(a)fakessh.eu sendmail_path = /usr/lib/sendmail } protocol sieve { managesieve_implementation_string = dovecot managesieve_logout_format = bytes ( in=%i : out=%o ) managesieve_max_line_length = 65536 } -- gpg --keyserver pgp.mit.edu --recv-key 092164A7 http://pgp.mit.edu:11371/pks/lookup?op=get&search=0x092164A7

1 0