- dovecot - dovecot.org

[Dovecot] Dovecot 2.0.9/1.2.11: crash in driver-pgsql.c
by dovecot＠excellency.fr 17 Feb '11

17 Feb '11

Hello, I finally took time to isolate and reproduce a long time Dovecot crash that is bugging me since at least Dovecot 1.0.x. It's still there in Dovecot 2.0.9, which I just tested. Here is the backtrace from my logs: Feb 16 16:56:57 dovecot: auth: Error: pgsql: Query timed out, aborting Feb 16 17:25:16 dovecot: auth: Panic: file driver-pgsql.c: line 349 (result_finish): assertion failed: (db->io == NULL) Feb 16 17:25:16 dovecot: auth: Error: Raw backtrace: /usr/lib/dovecot/libdovecot.so.0 [0x7fdbfe6d4272] -> /usr/lib/dovecot/libdovecot.so.0 [0x7fdbfe6d42d6] -> /usr/lib/dovecot/libdovecot.so.0(i_error+0) [0x7fdbfe6d46c8] -> dovecot/auth [0x42f555] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handle_timeouts+0xdd) [0x7fdbfe6df9bd] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_handler_run+0x5e) [0x7fdbfe6e05be] -> /usr/lib/dovecot/libdovecot.so.0(io_loop_run+0x28) [0x7fdbfe6df618] -> /usr/lib/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7fdbfe6cd883] -> dovecot/auth(main+0x2d4) [0x419474] -> /lib/libc.so.6(__libc_start_main+0xe6) [0x7fdbfc4f91a6] -> dovecot/auth [0x40da09] Feb 16 17:26:35 dovecot: master: Error: service(auth): child 140263 killed with signal 6 (core not dumped) Both my userdb and passdb use a PostgreSQL database. The bug happens when the PostgreSQL host suddenly becomes unavailable. I can easily reproduce it by launching Dovecot with my host being up, and then null routing its IP address (to simulate an outage): route add -host 10.0.0.2 reject (10.0.0.2 being the IP address of the PostgreSQL host.) >From that point, if I try to authenticate (using a POP3 connection, for instance), dovecot-auth will crash one or two minutes later. In Dovecot 2.0.9, despite the crash, authentication will resume once the PostgreSQL is back up (which can be simulated with: route delete -host 10.0.0.2 reject). In Dovecot 1.x however (latest version I tested is 1.2.11), it will stay in this state forever (logs are flooded with Password query failed: Not connected to database.) Core dump is available on http://paste.excellency.fr/dovecot.core # dovecot -n # 2.0.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.37 x86_64 Debian 5.0.6 auth_mechanisms = plain login auth_username_chars = disable_plaintext_auth = no first_valid_gid = 2000 first_valid_uid = 2000 listen = *, [::] log_timestamp = "%Y-%m-%d %H:%M:%S " mail_fsync = always mail_location = maildir:~/ mail_max_userip_connections = 100 mail_nfs_index = yes mail_nfs_storage = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date mmap_disable = yes passdb { args = /etc/dovecot/dovecot-pgsql.conf driver = sql } plugin { quota = maildir sieve = filter.sieve sieve_max_actions = 64 sieve_max_redirects = 100 } protocols = imap pop3 service auth { unix_listener auth-client { mode = 0666 } } service imap-login { service_count = 0 } service imap { process_limit = 1024 } service managesieve-login { service_count = 0 } service pop3-login { service_count = 0 } service pop3 { process_limit = 1024 } ssl_cert = </etc/ssl/certs/my.pem ssl_key = </etc/ssl/private/my.key userdb { args = /etc/dovecot/dovecot-pgsql.conf driver = sql } protocol imap { imap_client_workarounds = delay-newmail mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota } # cat /etc/dovecot/dovecot-pgsql.conf driver = pgsql connect = host=10.0.0.2 dbname=foobar user=dovecot password=foobar default_pass_scheme = CRYPT password_query = SELECT password FROM mailbox WHERE domain = '%d' AND local_part = '%n' user_query = SELECT home, uid, gid, system_user, quota_rule FROM mailbox WHERE domain = '%d' AND local_part = '%n' -- Cyril

3 2

[Dovecot] Cannot subscribe to shared folders
by Lukas Haase 17 Feb '11

17 Feb '11

Hi, I am unable to subscribe to shared folders with Thunderbird although everything seems to be correct (dovecot 1.2). What does not work? I click "Subscribe..." in Thunderbird. The shared mailboxes are listed, all with [ ] (no checkbox). Therefore I mark them [x], click "subscribe" and "OK". However, the folders do not appear in the folder tree. Furthermore, when I again click "Subscribe..." the folders do NOT have the mark [x] any more. This happens on different computers with *new* profiles. My configuration is the following (maybe that is the problem?!): namespace private { separator = . prefix =. inbox = no list = no hidden = yes location = maildir:~/Maildir-root subscriptions = yes } namespace private { separator = . prefix = INBOX. inbox = yes subscriptions = yes } namespace public { separator = . prefix = Shared. location = maildir:/var/mail/shared:INDEX=~/Maildir/shared:LAYOUT=fs subscriptions = no } The folder /var/mail/shared has appropriate permissions/form: * Owner: $user:mailuser (all users are member of "mailuser") * All files 0660, all folders 0770 * There exists a dovecot-shared in every folder * There exists a dovecot-acl in each folder where each user has the permissions "lrwstie" I switched on "mail_debug = yes" but no messages. However, the file ~/Maildir-root/subscriptions contains the subscribed folders: Shared.Group1 Shared.Group2 Shared.Group1/Test Furthermore, I switched on rawlog. Everything seems to be OK as well: .in-file: 2 subscribe "Shared.Group1" 3 lsub "" "INBOX.*" 5 lsub "" "Shared.*" 6 list "" "INBOX" .out-file: 2 OK Subscribe completed. [...] I am pretty sure it already worked. But it stopped working. Does anybody have a hint what could be the problem? Regards, Luke

2 4

[Dovecot] dsync conversion error
by Frank Bonnet 17 Feb '11

17 Feb '11

Hello When converting mailboxes I get that kind of error messages : dsync(xxx): Error: Next message unexpectedly lost from mbox file /user/xxx/mail/Sent at 9580036 (cached) the dsync command I run is dsync -u xxx mirror mbox:/user/xxx/mail:INBOX=/var/mail/xxx Dovecot mail_location configuration is : mail_location = mbox:~/MailSpace/Mailboxes:DIRNAME=Mbox-Messages:INDEX=~/Index:CONTROL=~/Kontroll Note that ONLY happen on users's "Sent" folder, never on any other folder Thanks for any advices.

2 2

[Dovecot] Some questions about mdbox
by Maria Arrea 16 Feb '11

16 Feb '11

Hello I am starting testing with dovecot 2.0.9. In my //etc/dovecot/dovecot.conf/ I have the following line: /mail_location = mdbox:/mydomain.org/%2.26Hn/%2.200Hn/%n:INDEX=/mydomain.org/indexes/%2.26Hn… These are my questions (a long mail): *1º How should I restore from backup with mdbox & separate indexes?* I have read in dovecot list archive about restoring the full user mailbox to a temp location and then run something like: /doveadm import mdbox:/tmp/restored_mailbox/ "" savedsince 2011-01-01 / In my schema indexes and mail stores are in separated paths. What (and where!) should I copy to the temp location and how do I restore it? *2º Disaster recovery: indexes & mail store mismatch* When creating backups, indexes and mail store are in different paths. My backup takes several hours, so indexes are copied BEFORE mail stores. Is the backup consistent? What would happens if I lose indexes? Can I still recover the messages? Reading the doc it seems I would lose all the message flags and all the messages would be in INBOX, is this right? *3º Performance tuning with mdbox* I have read in dovecot archive about using mdbox_rotate_size & mdbox_rotate_interval to balance between a lot of smaill files (like Maildir) or really big files (a "variant" of mbox). I have tested with a big vaule of mdbox_rotate_size (100 MB) and is a complete disaster for backup:if a user delete a message, then a nightly cronjob purges the mailbox (refcount=0) a new mdbox .m file is created!. So deleting a 1 k message from the mailbox of an user is a new 99 MB file to backup. Any hint about this? I am also using ext4 (rw,noatime,nodiratime,commit=15,data=writeback) and I have read about mdbox_preallocate_space. I will try it, any hint about this setup? This is not still in production, so I have no idea about IOPs and user concurrency, but I have several thousand users. *4º Compression with mdbox* I am reading about compression in mdbox (zlib/bzlib). I am starting with zlib, has anyone experience with bzlib? How much load is bzlib against zlib and how much delay adds to user experience (and space savings vs zlib???) ? *5º Recover a message with refcount=0* Is there any way to recover a message with refcount=0 without using a backup? I am thinking about the following plan: /1º a user gets her mail via pop3 (or delete a message via imap and empties trash) 2º the message is still stored in my mail server, but has refcount=0 3º nightly backup is performed 4º nightly cronjob purges the mailbox (deleting messaged with refcount=0)/ Using this schema the backup is bigger, but If I can recover files with refcount=0 from backup we can give our users a safeguard against accidentaly deleted pop3 mail. Thank you very much for your responsed and sorry for my (awful) english! Maria

3 2

Re: [Dovecot] Some questions about mdbox
by Maria Arrea 16 Feb '11

16 Feb '11

Hello everybody I have found that with bzip compression level 6 I have a better balance between compression and decompression speed with dovecot+mdbox, but still too slow compared with zlib. Does anybody have any other hint about my original questions (see below) Regards Maria Arrea ----- Original Message ----- From: Maria Arrea Sent: 02/15/11 09:16 PM To: dovecot(a)dovecot.org Subject: [Dovecot] Some questions about mdbox Hello I am starting testing with dovecot 2.0.9. In my //etc/dovecot/dovecot.conf/ I have the following line: /mail_location = mdbox:/mydomain.org/%2.26Hn/%2.200Hn/%n:INDEX=/mydomain.org/indexes/%2.26Hn… These are my questions (a long mail): *1º How should I restore from backup with mdbox & separate indexes?* I have read in dovecot list archive about restoring the full user mailbox to a temp location and then run something like: /doveadm import mdbox:/tmp/restored_mailbox/ "" savedsince 2011-01-01 / In my schema indexes and mail stores are in separated paths. What (and where!) should I copy to the temp location and how do I restore it? *2º Disaster recovery: indexes & mail store mismatch* When creating backups, indexes and mail store are in different paths. My backup takes several hours, so indexes are copied BEFORE mail stores. Is the backup consistent? What would happens if I lose indexes? Can I still recover the messages? Reading the doc it seems I would lose all the message flags and all the messages would be in INBOX, is this right? *3º Performance tuning with mdbox* I have read in dovecot archive about using mdbox_rotate_size & mdbox_rotate_interval to balance between a lot of smaill files (like Maildir) or really big files (a "variant" of mbox). I have tested with a big vaule of mdbox_rotate_size (100 MB) and is a complete disaster for backup:if a user delete a message, then a nightly cronjob purges the mailbox (refcount=0) a new mdbox .m file is created!. So deleting a 1 k message from the mailbox of an user is a new 99 MB file to backup. Any hint about this? I am also using ext4 (rw,noatime,nodiratime,commit=15,data=writeback) and I have read about mdbox_preallocate_space. I will try it, any hint about this setup? This is not still in production, so I have no idea about IOPs and user concurrency, but I have several thousand users. *4º Compression with mdbox* I am reading about compression in mdbox (zlib/bzlib). I am starting with zlib, has anyone experience with bzlib? How much load is bzlib against zlib and how much delay adds to user experience (and space savings vs zlib???) ? *5º Recover a message with refcount=0* Is there any way to recover a message with refcount=0 without using a backup? I am thinking about the following plan: /1º a user gets her mail via pop3 (or delete a message via imap and empties trash) 2º the message is still stored in my mail server, but has refcount=0 3º nightly backup is performed 4º nightly cronjob purges the mailbox (deleting messaged with refcount=0)/ Using this schema the backup is bigger, but If I can recover files with refcount=0 from backup we can give our users a safeguard against accidentaly deleted pop3 mail. Thank you very much for your responsed and sorry for my (awful) english! Maria

1 0

[Dovecot] Auth processes/login processes count
by Thomas Hummel 16 Feb '11

16 Feb '11

Hello Timo, I'm running dovecot-1.2.12/Maildir/LDAP-passdb/LDAP-prefetched-userdb and I'm experiencing "random" latency (which even seem to make squirrelmail timeout sometimes) at login without noticing particular load on the server. LDAP backends are running fine as far as I know so I was investigating maybe mistuned limits : Dovecot is configured (for ~3000 users) with . each login processed in its own process : #login_process_per_connection = yes #login_max_connections = 256 . counts as : #login_processes_count = 3 login_max_processes_count = 1024 auth default { [...] count = 5 } I've got several questions : 1) how can I count more than login_max_processes_count processes ? # ps aux | egrep -i '(pop3|imap)-login' | wc -l # 1031 2) could be my auth process count the bottleneck ? is there a rule of thumb to deduce it from the login process count ? 3) on dovecot login/master/auth design, is my understanding correct ? - dovecot-1.2.x : . passdb and userdb lookup are done by the auth process, on behalf of either the login process which wants to know if the client's password is correct or of the master process which wants to check if it was actually correct and which wants to have the userdb info. . client connection fd is sent to the master which creates the mail process and transfers the fd to it. - dovecot-2.x : . client connection fd does not go through the master anymore but through a socket in the login/ directory . it's the mail process which ask the auth process for the userdb lookup 4) with LDAP backend using {CRYPT} passwords dans with plaintext auth enabled in dovecot, is the clear password crypted before comparing it to the LDAP crypted password or does dovecot try to simply bind to LDAP with the clear password ? Thanks -- Thomas Hummel | Institut Pasteur <hummel(a)pasteur.fr> | Groupe Exploitation et Infrastructure

2 1

[Dovecot] TLS with autosigned certs
by Antonio Perez-Aranda 16 Feb '11

16 Feb '11

Is normal this info/error with autosigned certs? Feb 14 20:25:47 pop3-login: Info: Disconnected (no auth attempts):ip=[127.0.0.1], port=[51243], protocol=pop3, TLS handshaking: SSL_accept() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca: SSL alert number 48 I set on config a specific certs for pop3 protocol including key, cert and ca, protocol pop3 { ssl_cert = </etc/ssl/pop3d.cert ssl_key = </etc/ssl/pop3d.key ssl_ca = </etc/ssl/ca.pem } If I check as true confirmed on Thunderbird client, then, the error is no reported any more. -- Antonio Pérez-Aranda Alcaide aperezaranda(a)yaco.es Yaco Sistemas S.L. http://www.yaco.es/ C/ Rioja 5, 41001 Sevilla Teléfono +34 954 50 00 57 Fax +34 954 50 09 29

3 3

[Dovecot] test emails did not arrive at SMTP server : after dovecot installation
by sunhux G 16 Feb '11

16 Feb '11

Just set up postfix & it's running on my RHES 4.2 box. Immediately after postfix is up, I test sending emails from a permitted domain (ahhh, on this postfix server's domain firewall, we even have a firewall rule which permits Tcp25 from those few sending domains' SMTP servers) using an email client to sender_id@[IP_address_of_the_postfix_server] & the /var/log/maillog on the postfix server indicated the email arrives at the postfix server (with some errors though) : # grep recipient_id /var/log/maillog* maillog:Feb 15 11:41:52 hostname postfix/smtpd[6891]: NOQUEUE: reject: RCPT from gate1.mds.com.sg[203.126.130.157]: 554 5.7.1 <recipient_id(a)[202.6.163.31]>: Relay access denied; from=<prvs=020cae8c4=recipient_id(a)mds.com.sg> to=<recipient_id(a)[202.6.163.31]> proto=ESMTP helo=<gate1.mds.com.sg> maillog:Feb 15 13:43:20 hostname sendmail[7688]: NOQUEUE: SYSERR(recipient_id): can not chdir(/var/spool/mqueue/): Permission denied Then I installed dovecot rpm on my RHES box : uninstall it as it's an old version & reinstall with a newer version & start up dovecot as well. I did not test sending to sender_id@domain_name at that time because the domain I purchased from a domain provider/registrar has yet to be registered in our ISP's DNS. Subsequently I registered the following A, MX & NS records with our ISP : A: myportaltech.com. IN A 202.6.163.31 A: smtp.myportaltech.com. IN A 202.6.163.31 PTR: 31.163.6.202.in-addr.arpa. IN PTR smtp.myportaltech.com. MX: myportaltech.com. IN MX 10 smtp.myportaltech.com. NS: myportaltech.com. IN NS ns1.businessexprezz.com. NS: myportaltech.com. IN NS ns2.businessexprezz.com. The above myportaltech is just a fictitious name of my domain but I can provide the actual domain name if needed. After the above records have been propagated to all other DNSes, I test sending email from the same permitted domain, this time using domain name & the email never arrives & I did not receive a 'bounced mail' notification too. Then I test sending from the same domain to recipient_id(a)[202.6.163.31] & this time round, the test email never show up in /var/log/maillog* anymore. The network/security guys confirmed that the firewall logs did not show any denied SMTP records. So how do I go about troubleshooting this? Is this a DNS record entries issue, firewall/network issue, related to dovecot or something within my postfix server? Thanks Sun

2 1

[Dovecot] Dovecot can't start with error pop3-login & imap-login not supported protocol family
by sunhux G 16 Feb '11

16 Feb '11

I've done the installation (rpm downloaded from the RHES4 link given by Oli below) but dovecot won't start, it gave the error pop3-login & imap-login not supported by protocol. I'm sure with a much older version (in my previous post) of dovecot, I'm able to start up dovecot & it did not give this error. Thought dovecot is suppose to come with pop3 & imap? # rpm -Uvh --force mysqlclient14-4.1.22-1.el4s1.1.i386.rpm warning: mysqlclient14-4.1.22-1.el4s1.1.i386.rpm: V3 DSA signature: NOKEY, key ID 443e1821 Preparing... ########################################### [100%] 1:mysqlclient14 ########################################### [100%] /sbin/ldconfig: /usr/lib/libcpqlsptransport.so.0 is not a symbolic link [root@tmp]# rpm -qa | grep -i mysql mysql-4.1.12-3.RHEL4.1 perl-DBD-MySQL-2.9004-3.1 php-mysql-4.3.9-3.8 mysql-server-4.1.12-3.RHEL4.1 mysqlclient14-4.1.22-1.el4s1.1 <== it's there now [root@tmp]# rpm -i ./dovecot-2.0.9-1_125.el4.i386.rpm warning: only V3 signatures can be verified, skipping V4 signature warning: only V3 signatures can be verified, skipping V4 signature [root@tmp]# cd /etc [root@etc]# ls -ld rc*/*dove* lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc0.d/K01dovecot -> ../init.d/dovecot lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc1.d/K01dovecot -> ../init.d/dovecot lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc2.d/K01dovecot -> ../init.d/dovecot lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc3.d/K01dovecot -> ../init.d/dovecot lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc4.d/K01dovecot -> ../init.d/dovecot lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc5.d/K01dovecot -> ../init.d/dovecot lrwxrwxrwx 1 root root 17 Feb 16 11:37 rc6.d/K01dovecot -> ../init.d/dovecot [root@nspplss2 etc]# ls -ld rc*/S*dove* ls: rc*/S*dove*: No such file or directory [root@nspplss2 etc]# pwd /etc [root@etc]# ls rc*/*postf* rc0.d/K30postfix rc2.d/S80postfix rc4.d/S80postfix rc6.d/K30postfix rc1.d/K30postfix rc3.d/S99postfixstart rc5.d/S80postfix [root@etc]# ln -s /etc/init.d/dovecot /etc/rc3.d/S70dovecot [root@etc]# ln -s /etc/init.d/dovecot /etc/rc5.d/S70dovecot [root@]# init.d/dovecot start Starting Dovecot Imap: Error: socket() failed: Address family not supported by protocol Error: service(pop3-login): listen(::, 110) failed: Address family not supported by protocol Error: socket() failed: Address family not supported by protocol Error: service(pop3-login): listen(::, 995) failed: Address family not supported by protocol Error: socket() failed: Address family not supported by protocol Error: service(imap-login): listen(::, 143) failed: Address family not supported by protocol Error: socket() failed: Address family not supported by protocol Error: service(imap-login): listen(::, 993) failed: Address family not supported by protocol Fatal: Failed to start listeners [FAILED] [root@etc]# telnet localhost 110 Trying 127.0.0.1... telnet: connect to address 127.0.0.1: Connection refused telnet: Unable to connect to remote host: Connection refused [root@etc]# rpm -qa | grep dove dovecot-2.0.9-1_125.el4 [root@etc]# chkconfig --list | grep pop3 [root@etc]# chkconfig --list | grep imap On Tue, Feb 15, 2011 at 8:39 PM, Oli Schacher <dovecot(a)lists.wgwh.ch> wrote: > On Tue, 15 Feb 2011 18:00:53 +0800 > sunhux G <sunhux(a)gmail.com> wrote: > > > I'm on RHES 4.x > > > > I downloaded 2 dovecot RPM packages from > > http://packages.sw.be/dovecot/ > > > > dovecot-1.0.13-1.el4.rfx.i386.rpm 13-Nov-2010 22:15 1.8M > > RHEL4 and CentOS-4 x86 32bit > > and > > dovecot-0.99.13-1.2.el4.test.i386.rpm 21-Feb-2005 16:57 693K > > RHEL4 and CentOS-4 x86 32bit > > > > [...] > >Q1: > >Is the test version rpm of dovecot reliable? It's only half the size > >of the other rpm (that I was not able to install) > > 0.9 / 1.0 : These versions are way out of date and you will probably > have a hard time getting support. I suggest you try the current > versions from atrpms.net We have been using the 2.0 RHEL5 packages by > Axel Thimm for quite some time now and they work flawlessly. > > Link for RHEL4 packages: http://packages.atrpms.net/dist/el4/dovecot/ > > Maybe this fixes Q2 as well ;-) > > Q3: You'll find lots of examples / tutorials in the dovecot wiki : > http://wiki2.dovecot.org/#HOWTOs.2C_examples_and_tutorials > > HTH > > Oli >

2 2

[Dovecot] [Off Topic] - Any backup solutions ?
by Spyros Tsiolis 16 Feb '11

16 Feb '11

Hello kind people, OK, I know this is not for this list; However, reading all these mails on the topic of filesystems, I thought that it's a good place to ask about backup software on linux. Here's what I have. It's an HP 230 Ultrium LTO 1 SCSI backup mechanism (tape streamer to be exact). I'm supposed to hook it up today with an adaptec AHA-2940 controller to a gigabyte mainboard running Ubuntu Server 10.04. Does anyone have a config close to this one ? What kind of software would you chaps recommend ? dd ? Bacula ?? Other ?? This mechanism is supposed to grab backups from the same machine running it (the tape streamer), which is an ubuntu server version 10.04 with three SATA disks and maybe some stuff remotely (through the local network that is), from a couple of windows 7 / windows xp machines. Also 100Gb on every tape is not going to be enough and it looks like I will have to swap tapes during the process. Thank you all for your time and patience, Best Regards, spyros ---- "I merely function as a channel that filters music through the chaos of noise" - Vangelis

1 0