- dovecot - dovecot.org

[Dovecot] dovecot %u variable problem....
by Kris 26 Apr '12

26 Apr '12

Hi guys n gals, im on an ubuntu 11 VPS on linode (dovecot version 2.013). ive spent a lot of time and googling trying to get postfix, dovecot, ssl, saslauthd and mysql to all play nice together and im quite close now but theres something going on i cant quite explain and id really appreciate some help. I have been going for ten hours on this today and it seems in direct conflict with what dovecot themselves say... the problem is a variable in dovecot (%u) which is normally used as a full username in sql query strings (i.e. user@domain) and is simply dropping the domain for some reason... From dovecot docs (http://wiki2.dovecot.org/Variables) The variables that work everywhere are: %u user full username (e.g. user@domain) so i have postfix and stuff set up ok to send at least and i know this because ive tested it by hard-coding the string in the sql query myself with a successful send... but, anyway, the problem with the '%u' variable is it is doing stuff like this: (from mail.log and mysql.log) localhost dovecot: auth: Debug: sql(admin(a)microhard.com,xx.xx.xx.xx): SELECT home,uid,gid FROM users WHERE id = 'admin(a)microhard.com' this was derived from the following query in /etc/dovecot/dovecot-sql.conf.ext user_query = SELECT home,uid,gid FROM users WHERE id = '%u' so no problem there.... but then this happens when i attempt to send mail and this means the user gets rejected because dovecot cant find the user id in the db without the correct string (i.e. user@domain): localhost dovecot: auth: Debug: sql(admin,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin' Query SELECT id as user, crypt as password FROM users WHERE id= 'admin' which is derived from the following query also in /etc/dovecot/dovecot-sql.conf.ext password_query = SELECT id as user, crypt as password FROM users WHERE id= '%u' and to further confuse me, this 'unchangeable' variable sometimes decides to right itself without any obvious reason... localhost dovecot: auth: Debug: sql(admin(a)microhard.com,xx.xx.xx.xx): query: SELECT id as user, crypt as password FROM users WHERE id= 'admin(a)microhard.com' im wondering if its something to do with my client dropping the domain, or whether dovecot has an error, or some other thing ive overlooked... my dovecot.conf looks a lot like this: # 2.0.13: dovecot.conf # OS: Linux 3.0.18-x86_64-linode24 x86_64 Ubuntu 11.10 ext3 first_valid_uid = 5000 last_valid_uid = 5000 login_greeting = I likes my ducks n geese I do. mail_location = maildir:/var/spool/mail/virtual/%u #enables logging all failed authentication attempts. auth_verbose=yes #enables all authentication debug logging (also enables auth_verbose). Passwords are logged as <hidden>. auth_debug=yes #does everything that auth_debug=yes does, but it also removes password hiding. auth_debug_passwords=yes #enables all kinds of mail related debug logging, such as showing where Dovecot is looking for mails. mail_debug=yes #enables logging SSL errors and warnings. Even without this setting if connection is closed because of an SSL error, the error is logged as the disconnection reason (v1.1+). verbose_ssl=yes passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { acl = vfile:/etc/dovecot/acls #quota = maildir:storage=10240:messages=1000 trash = /etc/dovecot/trash.conf } protocols = " imap" service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { group = dovecot mode = 0660 user = dovecot } user = dovecot } ssl_ca = /etc/pki/dovecot/certs/ca-bundle.crt ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.pem userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } userdb { driver = prefetch } userdb { driver = passwd } protocol imap { mail_plugins = quota imap_quota } protocol pop3 { mail_plugins = quota pop3_client_workarounds = outlook-no-nuls oe-ns-eoh } protocol lda { info_log_path = /var/log/dovecot-deliver.log log_path = /var/log/dovecot-deliver.log mail_plugins = quota postmaster_address = admin(a)microhard.com } any help *much* appreciated! cheerz Kris

6 12

[Dovecot] dsync: convert only mbox in /var/mail to maildir?
by Jeff Simmons 26 Apr '12

26 Apr '12

I'm migrating an old UW pop server to a new Dovecot server. The old server did not allow mail to be stored on the server, so only mbox files exist in /var/mail, with nothing in /home/$user. I'd like to copy the /var/mail directory over to the new server and then use dsync to convert the mbox files and put them in /home/$user/Maildir, but every permutation of the command line I've tried dies with a variation on can't find/write either /var/mail/mail or /home/$user/mail, seems it won't deal with mbox files without an associated mail directory. Can dsync be used for this kind of conversion, or should I look elsewhere for conversion utilities? Thanks for helping a noob who can't traverse a wiki without leaving a trail of breadcrumbs. -- Jeff Simmons jeff(a)j-simmons.net Simmons Consulting - Network Engineering, Administration, Security

2 2

[Dovecot] 2.1.x changes and dsync -- Wiki suggestion
by Jeff Kletsky 26 Apr '12

26 Apr '12

There are a couple things that show up on the Upgrading/2.1 wiki page that would be useful to have on pages describing dsync (Tools/Dsync, Migration/MailFormat, Tools, perhaps others): * dsync was merged into doveadm * You can't dsync 2.0.x to/from 2.1.x That second point is especially "interesting" to me as it seems that Apple MacOS X Server is still on 2.0.16, at least from what I see on opensource.apple.com. I was planning on migrating mail off the current server onto my friend's new MacMini, but I'll have to figure out how to "downgrade" the mail jail to a 2.0 version, it seems. Thanks for all the great work, Jeff

1 0

[Dovecot] Reading Mutt mboxes from Thunderbird
by Leonardo M. Ramé 26 Apr '12

26 Apr '12

Hi, I'm a user of Mutt, and before moving to Thunderbird I'm trying to share mboxes between them. To do this, I installed Dovecot to create an IMAP server in my local machine, to serve mutt mboxes to be read by Thunderbird. So far, I can read my inbox from /var/mail/%u, but in mutt, I have many folders in ~/mail, like 2010-mails, 2011-mails, mailing-list-x, mailing-list-y, and so on. How can I configure Dovecot to read them?. In /etc/dovecot/conf.d/10-mail.conf I have this: mail_location = mbox:~/Mail:INBOX=/var/mail/%u I'm using Ubuntu 11.10, and Dovecot 2.0.13. Regards, -- Leonardo M. Ramé Medical IT - Griensu S.A. Av. Colón 636 - Piso 8 Of. A X5000EPT -- Córdoba Tel.: +54(351)4246924 +54(351)4247788 +54(351)4247979 int. 19 Cel.: +54(351)156629292

3 2

[Dovecot] How to remove duplicate GUID messages from storage?
by Daniel Parthey 26 Apr '12

26 Apr '12

Hi, I tried to import messages like this: doveadm import -u username(a)example.org mdbox:/mail/dovecot/example.org/username/mail mbox:/root/mail.txt all This seemed to have imported the same messages in the mail storage, as they were already there. And now I have got a lot of duplicates. Then I deleted the mbox:/root/mail mail stuff from "mailboxes" folder and the storage remained, with duplicate GUIDs. The following command doveadm -v force-resync -u username(a)example.org INBOX outputs lots of messages like this one, with different GUID doveadm(username(a)example.org): Error: mdbox /mail/dovecot/example.org/username/mail/storage: Duplicate GUID 0b8a032d66a0924fb42c0000de5f8128 in m.55:45484041 and m.14:52173045 The messages at m.55:45484041 and m.14:52173045 have the same content, since they were seemingly imported from the mail store itself, they are in the storage twice now and require twice as much disk space as before. How can I manually remove these identical, duplicate messages from the storage to save space? Dovecot does not do it automatically. Kind regards, Daniel

2 3

[Dovecot] dovecot-lda error
by Hadi Salem 26 Apr '12

26 Apr '12

Hi, im getting the fallowing error. from dovecot-lda log Apr 25 17:30:41 lda(jamal(a)example.com): Fatal: setgid(5003(vmail) from userdb lookup) failed with euid=8(mail), gid=12(mail), egid=12(mail): Operation not permitted (This binary should probably be called with process group set to 5003(vmail) instead of 12(mail)) this error occur if im using different uid and gid part from mail uid.. jamal@example.com:{PLAIN}123:5003:5003::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir but if i use mail uid it work fine.. jamal@example.com:{PLAIN}123:8:12::/var/vmail/example.com/jamal/::userdb_mail=maildir:~/Maildir i tried using multiple uids as mention in wiki2.dovecot.org/LDA bye setting up setuid-root for dovecot-lda but im getting on-there error from sendmail :- Apr 25 17:30:41 host1 sendmail[31024]: q3PEUen2031022: to=<jamal(a)example.com>, delay=00:00:00, xdelay=00:00:00, mailer=dovecot, pri=120359, relay=dovecot, dsn=4.0.0, stat=Deferred: dovecot mailer (/usr/local/libexec/dovecot/dovecot-lda) exited with EX_TEMPFAIL im using sendmail. [root@host1 conf.d]# doveconf -n # 2.0.20: /usr/local/etc/dovecot/dovecot.conf # OS: Linux 2.6.18-164.el5 x86_64 CentOS release 5.4 (Final) auth_debug = yes auth_debug_passwords = yes auth_verbose = yes auth_verbose_passwords = plain debug_log_path = /var/log/dovecot.log disable_plaintext_auth = no first_valid_gid = 12 first_valid_uid = 8 hostname = host1.bigmama.com info_log_path = /var/log/dovecot.log last_valid_gid = 50000 last_valid_uid = 50000 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = mail mail_location = maildir:~/Maildir mail_uid = mail passdb { args = scheme=PLAIN username_format=%u /etc/dovecot/users driver = passwd-file } postmaster_address = root(a)host1.bigmama.com service auth { unix_listener auth-userdb { group = mail mode = 0666 user = mail } } ssl = no userdb { args = username_format=%u /etc/dovecot/users driver = passwd-file } protocol lda { info_log_path = /var/log/dovecot-lda.log log_path = /var/log/dovecot-lda-errors.log }

2 1

[Dovecot] 2.1.5 problem with mail_location
by postmaster 25 Apr '12

25 Apr '12

I've been advised to set : mail_location = maildir:~/ as the home directory is found through a sql request. this has been working perfectly up to and including 2.1.4 from 2.1.5 however this setting gives cause to a problem : instead of looking in the mail home eg /var/mail/domain/user\@domain/ dovecot is looking for mail in /var/mail/domain/user\@domain/~ the '~' directory is created upon the first request for mail and all future mail transactions in dovecot is done relative to this folder. is this behavior a bug or a feature ??? -- Bjarne D Mathiesen København N ; Danmark ; Europa ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ denne besked er skrevet i et totalt M$-frit miljø MacOS X 10.7.3 Lion ; 2.8GHz Intel Core i7 ; 16GB 1067MHz DDR3

2 1

[Dovecot] sieve vacation redirect problem
by Robert Schetterer 25 Apr '12

25 Apr '12

Hi , since upgrade sieve vacation, redirect stopped working getting postfix error while sending end of data -- message may be sent more than once i played a lot about sieve , lda submission host conf entries etc but all failed , i am not seeing any heavy bug in the logs -- Best Regards MfG Robert Schetterer Germany/Munich/Bavaria

4 15

[Dovecot] Error compiling pigeonhole 0.3.0 with dovecot-2.1.5 in openbsd 5.0
by Joseba Torre 25 Apr '12

25 Apr '12

Hi, I've just tried to compile dovecot & pigeonhole in my openbsd box, and got this: gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-quote.o -MD -MP -MF .deps/managesieve-quote.Tpo -c -o managesieve-quote.o managesieve-quote.c mv -f .deps/managesieve-quote.Tpo .deps/managesieve-quote.Po gcc -DHAVE_CONFIG_H -I. -I../.. -I/home/seajob/build/dovecot-2.1.5 -I/home/seajob/build/dovecot-2.1.5/src/lib -I/home/seajob/build/dovecot-2.1.5/src/lib-dict -I/home/seajob/build/dovecot-2.1.5/src/lib-dns -I/home/seajob/build/dovecot-2.1.5/src/lib-mail -I/home/seajob/build/dovecot-2.1.5/src/lib-imap -I/home/seajob/build/dovecot-2.1.5/src/lib-fs -I/home/seajob/build/dovecot-2.1.5/src/lib-charset -I/home/seajob/build/dovecot-2.1.5/src/lib-index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index -I/home/seajob/build/dovecot-2.1.5/src/lib-storage/index/raw -I../.. -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -I/usr/include -MT managesieve-parser.o -MD -MP -MF .deps/managesieve-parser.Tpo -c -o managesieve-parser.o managesieve-parser.c managesieve-parser.c: In function 'quoted_string_istream_read': managesieve-parser.c:639: error: 'EPROTO' undeclared (first use in this function) managesieve-parser.c:639: error: (Each undeclared identifier is reported only once managesieve-parser.c:639: error: for each function it appears in.) *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src/lib-managesieve (line 92 of /usr/share/mk/sys.mk). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0/src (line 289 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 474 of Makefile). *** Error code 1 Stop in /home/seajob/build/dovecot-2.1-pigeonhole-0.3.0 (line 353 of Makefile).

2 2

[Dovecot] dsync on large mailbox "fails"
by Jeff Kletsky 25 Apr '12

25 Apr '12

I've been trying to convert an existing set of mbox mail to maildir using dsync, but it seems to fail when it gets to the large boxes. $ dsync -Dv mirror maildir:~/Maildir Everything seems to work reasonably well until the first of the large boxes hits: dsync(jeff): Warning: Maildir /home/jeff/Maildir/.cron.2010-07-10: Synchronization took 1210 seconds (102289 new msgs, 0 flag change attempts, 0 expunge attempts) dsync(jeff): Warning: Transaction log file /home/jeff/Maildir/.cron.2010-07-10/dovecot.index.log was locked for 1210 seconds After this, the doveadm process goes from ~75% of the CPU core utilization to 100% and stays there, even overnight. I've also tried $ dsync -Dv backup maildir:~/Maildir with similar results. While I did see some threads on dsync speed, they seemed to be minutes for a few GB of data, not hours. There is plenty of space on the drive, and, from what I understand, there is not a file-count limit on FreeBSD's ufs (at least not one that I'm anywhere near) Any suggestions? Thanks! Jeff [jeff@mail ~]$ du -h -d 0 ~/mail 3.9G /home/jeff/mail [jeff@mail ~]$ ls -lh /var/mail/jeff -rw------- 1 jeff jeff 554M Apr 23 11:27 /var/mail/jeff [jeff@mail ~]$ df -h Filesystem Size Used Avail Capacity Mounted on /dev/ada0s2g 38G 18G 16G 53% / (Note that the hostname and the location of the ssl certificate has been manually edited for this message in the following output) From dmesg.boot: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.0-STABLE #0: Sat Apr 21 17:13:07 PDT 2012 root@my.host.name:/usr/obj/usr/src/sys/GENERIC i386 CPU: Intel(R) Atom(TM) CPU 330 @ 1.60GHz (1600.03-MHz 686-class CPU) Origin = "GenuineIntel" Id = 0x106c2 Family = 6 Model = 1c Stepping = 2 Features=0xbfe9fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> Features2=0x40e31d<SSE3,DTES64,MON,DS_CPL,TM2,SSSE3,CX16,xTPR,PDCM,MOVBE> AMD Features=0x20100000<NX,LM> AMD Features2=0x1<LAHF> TSC: P-state invariant, performance statistics real memory = 2147483648 (2048 MB) avail memory = 2077757440 (1981 MB) [jeff@mail ~]$ doveconf -n # 2.1.3: /usr/local/etc/dovecot/dovecot.conf # OS: FreeBSD 9.0-STABLE i386 auth_mechanisms = plain login disable_plaintext_auth = no first_valid_gid = 1000 first_valid_uid = 1000 listen = * mail_location = mbox:~/mail/:INBOX=/var/mail/%u mail_privileged_group = mail namespace { inbox = yes location = prefix = separator = / } passdb { args = session=yes dovecot driver = pam } protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-master { mode = 0600 } user = root } service imap-login { inet_listener imap { port = 0 } process_limit = 200 } ssl = required ssl_cert = </path/to/certificate.pem ssl_key = </path/to/certificate.pem userdb { args = blocking=yes driver = passwd } verbose_proctitle = yes protocol imap { imap_client_workarounds = tb-extra-mailbox-sep }

2 5