- dovecot - dovecot.org

[Dovecot] Fwd: Re: Dotlock dovecot-uidlist errors / NFS / High Load
by list＠airstreamcomm.net 21 Jan '11

21 Jan '11

Stan, Thanks for the reply. In our case we have actually already done most of the work you suggested to no avail. We had rebuilt two new ntp servers that sync against two stratum 1 sources, and all our nfs clients, regardless of using dovecot, sync to those two machines. You bring up the difference between bare metal and hypervisor, and we are running these machines on vmware 4.0. All the vmware knowledge base articles tend to push us towards ntp, and since we are using centos 5.5 there are no kernel modifications that need to be made regarding timing from what we can find. I will give the ntpdate option a try and see what happens. I was also hoping to understand why the uidlist file is the only file that uses dotlock, or if there was plans to give it the option to use other locking mechanisms in the future. Thanks again! Michael -------- Original Message -------- Subject: Re: [Dovecot] Dotlock dovecot-uidlist errors / NFS / High Load Date: Thu, 20 Jan 2011 10:57:24 -0600 From: Stan Hoeppner <stan(a)hardwarefreak.com> To: dovecot(a)dovecot.org list(a)airstreamcomm.net put forth on 1/20/2011 8:32 AM: > Secondly we thought the issues were due to NTP as the time stamps vary so > widely, so we rebuilt our NTP servers and found closer stratum 1 source > clocks to synchronize to hoping it would alleviate the problem but the > dotlock errors returned after about 12 hours. We have fcntl locking set in > our configuration file, but it is our understanding from look at the source > code that this file is locked with dotlock. > > Any help troubleshooting is appreciated. >From your description it sounds as if you're ntpd syncing each of the 4 servers against an external time source, first stratum 2/3 sources, then stratum 1 sources in an attempt to cure this problem. In a clustered server environment, _always_ run a local physical box/router ntpd server (preferably two) that queries a set of external sources, and services your internal machine queries. With RTTs all on your LAN, and using the same internal time sources for every query, this clock drift issue should be eliminated. Obviously, when you first set this up, stop ntpd and run ntpdate to get an initial time sync for each cluster host. If after setting this up, and we're dealing with bare metal cluster member servers, then I'd guess you've got a failed/defective clock chip on one host. If this is Linux, you can work around that by changing the local time source. There are something like 5 options. Google for "Linux time" or similar. Or, simply replace the hardware--RTC chip, mobo, etc. If any of these cluster members are virtual machines, regardless of hypervisor, I'd recommend disabling using ntpd, and cron'ing ntpdate to run once every 5 minutes, or once a a minute, whatever it takes to get the times to remain synced, against your local ntpd server mentioned above. I got to the point with VMWare ESX that I could make any Linux distro VM of 2.4 or 2.6 stay within one minute a month before needing a manual ntdate against our local time source. The time required to get to that point is a total waste. Cron'ing ntpdate as I mentioned is the quick, reliable way to solve this issue, if you're using VMs. -- Stan

3 3

Re: [Dovecot] Problems with Upgrade from Courier
by Aaron Pettitt 21 Jan '11

21 Jan '11

I was looking at my dovecot.deliver log and it's showing that it's delivering it to the Inbox: deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: auth input: home=/home/vmail/mybridemail.com/samantha.freeze deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: maildir: data=/home/vmail/mybridemail.com/samantha.freeze/ deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: maildir++: root=/home/vmail/mybridemail.com/samantha.freeze, index=, control=, inbox=/home/vmail/mybridemail.com/samantha.freeze deliver(samantha.freeze(a)mybridemail.com): 01/20/2011 10:44:27 Info: msgid=<001a01cbb8b8$eabe89c0$c03b9d40$@net>: saved mail to INBOX However, if I login as her through Telnet, it shows that she has no mail: * FLAGS (\Answered \Flagged \Deleted \Seen \Draft) * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft \*)] Flags permitted. * 0 EXISTS * 0 RECENT * OK [UIDVALIDITY 1295474980] UIDs valid * OK [UIDNEXT 1] Predicted next UID b OK [READ-WRITE] Select completed. If I look in the new folder under her folder, it shows the last emails I sent this morning: -rw------- 1 vmail vmail 3564 Jan 20 10:21 1295536875.M679042P20187.mybridemail.com,W=3672 -rw------- 1 vmail vmail 3540 Jan 20 10:27 1295537272.M522196P26548.mybridemail.com,W=3649 -rw------- 1 vmail vmail 3554 Jan 20 10:39 1295537952.M462095P9353.mybridemail.com,W=3662 -rw------- 1 vmail vmail 3540 Jan 20 10:44 1295538267.M893549P15392.mybridemail.com,W=3649 It's really strange why dovecot can deliver the mail to the inbox but cannot see the inbox when trying to retrieve the mail???? From: Aaron Pettitt [mailto:apettitt@comcast.net] Sent: Thursday, January 20, 2011 10:02 AM To: 'dovecot(a)dovecot.org' Subject: Problems with Upgrade from Courier I inherited a server from a previous employee. The server crashed so it was time to move everything over to another server. We have a web mail site and I installed everything running dovecot, postfix and roundcube. After I installed it, everything worked great when I created a new user. The new user could send and receive emails with no issues. I then copied the home directory over from the other server and ran the courier-dovecot migration script. It created the subscription files and the dovecot-uidlist files in each user (about 1000 total users). When I login as one of the existing users, it says that there is no mail in the mailbox. However, if I look at the user's cur and new folders, there is mail in those folders. If I send a new mail to the user, it does not show up in their inbox. If I look in their new folder, the new mail that I sent was delivered to that folder but it does not show up in their inbox. I've tried going to dovecot directly through telnet with the same results. I've been stuck for 2 days now so any help is greatly appreciated. Below is my dovecot.conf with all the comments removed. Thanks all! protocols = imap imaps disable_plaintext_auth = no log_path = '/var/log/dovecot/error.log' info_log_path = '/var/log/dovecot/info.log' log_timestamp = "%m/%d/%Y %H:%M:%S " #mail_location = maildir:~/ mail_location = maildir:/home/vmail/%d/%n/ mail_privileged_group = mail mail_debug = yes protocol imap { } protocol pop3 { } protocol managesieve { sieve_storage=~/sieve } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster(a)mybridemal.com mail_plugins = cmusieve global_script_path = /home/vmail/globalsieverc } auth_verbose = yes auth_debug = no auth_debug_passwords = no auth default { passdb sql { args = /etc/dovecot/dovecot-sql.conf } args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } user = root path = /var/run/dovecot/auth-master mode = 0600 user = vmail #group = } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } }

2 4

[Dovecot] Populating mailbox dir
by Mauricio Tavares 20 Jan '11

20 Jan '11

Let's say you created a (virtual) mail user but have not created the user mail directory. When you first try to access it, the mail directory is created and populated: ls -a /var/spool/vmail/domain.com/raub/ . dovecot.index.log dovecot-uidvalidity.4d375a39 tmp .. dovecot-uidlist new .Trash cur dovecot-uidvalidity subscriptions Who populates/creates the initial files and folders in the user mailbox? Dovecot? The mail client accessing it through imap? Elvis? If dovecot, is there a way to customize it a bit, so a few extra folders will also be thrown there?

3 2

[Dovecot] Imap Error
by Jason Liedtke 20 Jan '11

20 Jan '11

This morning I have a Outlook 2007 user who getting the error and I am unsure how to fix it. Cannot open this item. The server responded: "Error in IMAP command UID FETCH: Invalid uidset' DoveCot v 1.2.9 # 1.2.9: /etc/dovecot/dovecot.conf # OS: Linux 2.6.32-27-generic-pae i686 Ubuntu 10.04.1 LTS log_timestamp: %Y-%m-%d %H:%M:%S protocols: imap imaps pop3 pop3s managesieve listen(default): * listen(imap): * listen(pop3): * listen(managesieve): *:2000 disable_plaintext_auth: no login_dir: /var/run/dovecot/login login_executable(default): /usr/lib/dovecot/imap-login login_executable(imap): /usr/lib/dovecot/imap-login login_executable(pop3): /usr/lib/dovecot/pop3-login login_executable(managesieve): /usr/lib/dovecot/managesieve-login mail_privileged_group: mail mail_location: maildir:~/ mmap_disable: yes mail_nfs_storage: yes mail_nfs_index: yes mbox_write_locks: fcntl dotlock mail_executable(default): /usr/lib/dovecot/imap mail_executable(imap): /usr/lib/dovecot/imap mail_executable(pop3): /usr/lib/dovecot/pop3 mail_executable(managesieve): /usr/lib/dovecot/managesieve mail_plugins(default): quota imap_quota mail_plugins(imap): quota imap_quota mail_plugins(pop3): quota mail_plugins(managesieve): mail_plugin_dir(default): /usr/lib/dovecot/modules/imap mail_plugin_dir(imap): /usr/lib/dovecot/modules/imap mail_plugin_dir(pop3): /usr/lib/dovecot/modules/pop3 mail_plugin_dir(managesieve): /usr/lib/dovecot/modules/managesieve imap_client_workarounds(default): tb-extra-mailbox-sep imap_client_workarounds(imap): tb-extra-mailbox-sep imap_client_workarounds(pop3): imap_client_workarounds(managesieve): pop3_client_workarounds(default): pop3_client_workarounds(imap): pop3_client_workarounds(pop3): outlook-no-nuls oe-ns-eoh pop3_client_workarounds(managesieve): managesieve_logout_format(default): bytes=%i/%o managesieve_logout_format(imap): bytes=%i/%o managesieve_logout_format(pop3): bytes=%i/%o managesieve_logout_format(managesieve): bytes ( in=%i : out=%o ) namespace: type: private separator: / inbox: yes list: yes subscriptions: yes lda: postmaster_address: hostname: auth_socket_path: /var/run/dovecot/auth-master mail_plugins: quota sieve log_path: info_log_path: syslog_facility: mail auth default: username_format: %Lu passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf passdb: driver: sql args: /etc/dovecot/dovecot-sql.conf passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf passdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: sql args: /etc/dovecot/dovecot-sql.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf userdb: driver: ldap args: /etc/dovecot/dovecot-ldap.conf socket: type: listen master: path: /var/run/dovecot/auth-master mode: 432 user: vmail group: vmail plugin: quota: maildir quota_rule: *:bytes=20M sieve: ~/sieve/.dovecot.sieve sieve_dir: ~/sieve sieve_global_path: /var/mail/default.sieve sieve_before: /var/mail/sieve/global sieve_extensions: +imapflags

2 1

[Dovecot] ldap auth error
by pch0317 20 Jan '11

20 Jan '11

Hi, I have dovecot 2.0.beta6 and I'm newbie with dovecot. I try authenticate user via LDAP (MS ActiveDirectory). I have errors in log: mail dovecot: auth: Error: LDAP: binding failed (dn cn=administrator,ou=users,dc=my,dc=domain): Invalid credentials, 80090308: LdapErr: DSID-0C090334, comment: AcceptSecurityContext error, data 525, vece This is my dovecot.conf file: protocols = imap disable_plaintext_auth = no listen = * ssl = no auth_mechanisms = plain base_dir = /var/run/dovecot/ mail_location = mbox:/home/vmail/%n/ passdb { driver = ldap args =/etc/dovecot/dovecot-ldap.conf } userdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf } And this is my dovecot-ldap.conf: hosts = 192.168.0.19 dn = cn=administrator,ou=users,dc=my,dc=domain dnpass = password base = ou=myuser,dc=my,dc=domain scope = subtree ldap_version = 3 auth_bind = yes auth_bind_userdn = ou=myuser,dc=my,dc=domain user_attrs = homeDirectory=home,uidNumber=uid,gidNumber=gid user_filter = (&(objectClass=posixAccount)(uid=%u)) pass_attrs = uid=user,userPassword=password pass_filter = (&(objectClass=posixAccount)(uid=%u)) Should I have any file in /etc/pam.d ? Thanks

3 4

[Dovecot] domain stripping
by PA 20 Jan '11

20 Jan '11

Hi, using dovecot 2.0 and I'm using a couple of user DBs, sql/prefetch and pam. Currently if the user logins with username@domain it authenticates off the sql DB and works fine. If the user logins with username with no @domain it fails on the sql lookup and succeeds on the pam user DB. However I was wondering if I can have another sql DB lookup that says when the user logins and fails against the first two user DBs, sql/pam db, to try this last sql user DB and appends a default domain to it, because the sql DB lists username with the domain. Currently I have all users login in with no realm on the older mail server and I wanted to migrate these users to dovecot 2.x with minimal impact and wanted to have the ability for these virtual users to login with and without a realm. Thanks paul.

2 2

[Dovecot] Problems with Upgrade from Courier
by Aaron Pettitt 20 Jan '11

20 Jan '11

I inherited a server from a previous employee. The server crashed so it was time to move everything over to another server. We have a web mail site and I installed everything running dovecot, postfix and roundcube. After I installed it, everything worked great when I created a new user. The new user could send and receive emails with no issues. I then copied the home directory over from the other server and ran the courier-dovecot migration script. It created the subscription files and the dovecot-uidlist files in each user (about 1000 total users). When I login as one of the existing users, it says that there is no mail in the mailbox. However, if I look at the user's cur and new folders, there is mail in those folders. If I send a new mail to the user, it does not show up in their inbox. If I look in their new folder, the new mail that I sent was delivered to that folder but it does not show up in their inbox. I've tried going to dovecot directly through telnet with the same results. I've been stuck for 2 days now so any help is greatly appreciated. Below is my dovecot.conf with all the comments removed. Thanks all! protocols = imap imaps disable_plaintext_auth = no log_path = '/var/log/dovecot/error.log' info_log_path = '/var/log/dovecot/info.log' log_timestamp = "%m/%d/%Y %H:%M:%S " #mail_location = maildir:~/ mail_location = maildir:/home/vmail/%d/%n/ mail_privileged_group = mail mail_debug = yes protocol imap { } protocol pop3 { } protocol managesieve { sieve_storage=~/sieve } protocol lda { log_path = /home/vmail/dovecot-deliver.log auth_socket_path = /var/run/dovecot/auth-master postmaster_address = postmaster(a)mybridemal.com mail_plugins = cmusieve global_script_path = /home/vmail/globalsieverc } auth_verbose = yes auth_debug = no auth_debug_passwords = no auth default { passdb sql { args = /etc/dovecot/dovecot-sql.conf } args = uid=5000 gid=5000 home=/home/vmail/%d/%n allow_all_users=yes } user = root path = /var/run/dovecot/auth-master mode = 0600 user = vmail #group = } client { path = /var/spool/postfix/private/auth mode = 0660 user = postfix group = postfix } } } }

1 0

[Dovecot] Bug found, assertion failed
by Tobias Daucher 20 Jan '11

20 Jan '11

Hi there, We're running dovecot 2.06, with mdbox. Following message was in our syslog: Jan 20 09:26:48 servername dovecot: [ID 583609 mail.crit] imap(user): Panic: file istream-limit.c: line 79: assertion failed: (v_offset <= lstream->v_size) The problem could be solved on client side, by just deleting the ImapMail-Folder in Thunderbird. Why? Thunderbird tried to move a message, that obviously wasn't there. Dovecot got killed and the message above was in the syslog. Connection was closed and Thunderbird told me the server is dead. Thunderbird tried this every few seconds, and there was no way to say thunderbird "stop try moving". So the only way was to delete thunderbirds mail cache. I think it would be very nice, if dovecot doesn't die, just because the client tries to move a message, which isn't there. Thanks, Tobias Daucher -- Dr. Nagler & Company GmbH Hauptstraße 9 92253 Schnaittenbach Tel : 09622-7197-38 Fax : 09622-7197-50 Web : http://www.nagler-company.com E-Mail : tobias.daucher(a)nagler-company.com Hauptsitz: Schnaittenbach Handelregister: Amberg HRB 4653 Gerichtsstand: Amberg Steuernummer: 201/118/51825 USt.-ID-Nummer: DE 273143997 Geschäftsführer: Dr. Martin Nagler, Dr. Dr. Karl-Kuno Kunze

1 0

Re: [Dovecot] dovecot Digest, Vol 93, Issue 41
by Alan Brown 20 Jan '11

20 Jan '11

> From: Stan Hoeppner <stan(a)hardwarefreak.com> > Subject: Re: [Dovecot] SSD drives are really fast running Dovecot > > > Yes. Go with a cluster filesystem such as OCFS or GFS2 and an inexpensive SAN > storage unit that supports mixed SSD and spinning storage such as the Nexsan > SATABoy with 2GB cache: http://www.nexsan.com/sataboy.php I can't speak for OCFS2, but after several years' experience with the filesystem I strongly recommend NOT using GFS/GFS2. Its locking model is incredibly slow (500 locks/second on a filesystem mounted with quotas enabled and noatime) and results in dire performance - plus there's a known crash vulnerability if files are repeatedly renamed in large directories (this bites us regularly...) GFS2 isn't an "enterprise" filesystem by any stretch of the imagination, despite what a number of enthusiastic salespeople might try to convince you of. We're lucky to keep the GFS servers up for more than a week at a time.

3 2

[Dovecot] Upgrading from 1.2 to 2.0
by list＠airstreamcomm.net 20 Jan '11

20 Jan '11

Greetings Dovecot mailing list! We are an ISP that recently migrated from Courier to Dovecot, and are looking at the next round of upgrades. Hoping that we could get pointed to some documentation or get a general list of best practices for an upgrade from 1.2.13 to 2.0.9. Our current environment consists of four Dovecot machines that share an NFS backend hosting POP and IMAP. Our biggest question is whether we can support a rolling upgrade from 1.2.13 to 2.0.9, or if it would need to be an all for one cut. Thanks in advance.

3 2