- dovecot - dovecot.org

Rebuild UIDs for mdbox
by Darac Marjal 28 Nov '16

28 Nov '16

Hello all, I think I'm hitting an issue with overly large UIDs in one of my mailboxes. I initially thought it was a problem with claws-mail hitting an infinite loop. Claws-mail /seems/ to work fine, until it gets to this one mailbox, wherein the following is logged: [20:23:04] IMAP> 86 SELECT LocalCron [20:23:04] IMAP< * OK [CLOSED] Previous mailbox closed. [20:23:04] IMAP< * FLAGS (\Answered \Flagged \Deleted \Seen \Draft Junk NonJunk) [20:23:04] IMAP< * OK [PERMANENTFLAGS (\Answered \Flagged \Deleted \Seen \Draft Junk NonJunk \*)] Flags permitted. [20:23:04] IMAP< * 399 EXISTS [20:23:04] IMAP< * 0 RECENT [20:23:04] IMAP< * OK [UNSEEN 399] First unseen. [20:23:04] IMAP< * OK [UIDVALIDITY 1163720154] UIDs valid [20:23:04] IMAP< * OK [UIDNEXT 3069273506] Predicted next UID [20:23:04] IMAP< * OK [HIGHESTMODSEQ 120748] Highest [20:23:04] IMAP< 86 OK [READ-WRITE] Select completed (0.000 + 0.000 secs). imap-thread.c:1344:imap select run - end 0 imap-thread.c:388:generic_cb imap-thread.c:372:found imap 0x22c8f60 imap-thread.c:1431:imap select - end imap.c:3961:select: exists 399 recent 0 expunge 0 uid_validity 1163720154 can_create_flags 1 imap-thread.c:2018:imap fetch_uid - begin imap-thread.c:372:found imap 0x22c8f60 imap-thread.c:372:found imap 0x22c8f60 [20:23:04] IMAP- [fetching UIDs...] [20:23:04] IMAP> 87 UID FETCH 1:* (UID) [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< FETCH (UID 3069273244) [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< TCH (UID 3069273313) [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< TCH (UID 3069273382) [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< TCH (UID 3069273451) [20:23:04] IMAP< [FETCH data - 1024 bytes] [20:23:04] IMAP< [FETCH data - 551 bytes] imap-thread.c:2008:imap fetch_uid run - end 0 imap-thread.c:388:generic_cb imap-thread.c:372:found imap 0x22c8f60 imap-thread.c:2029:imap fetch_uid - end imap.c:542:unlocking session 0x22e4c00 imap.c:4663:get_num_list: got 399 msgs imap.c:4673:removing old messages from /home/darac/.claws-mail/imapcache/mail.darac.org.uk/darac(a)darac.org.uk/LocalCron imap.c:4677:get_num_list - ok - 399 folder.c:2265:Removed message 2147483647 from cache. That last line (Removed message 2147483647 from cache) is repeated ad infinitum. Now, I happened to spot that that number is 2^31 - 1 so started looking to see if claws-mail was handling UIDs incorrectly. That lead me to http://wiki.dovecot.org/Clients/NegativeUIDs. Firstly, that page states that UIDs should be 32-bit integers. If so, why do there appear to be UIDs greater than 3 billion in the mailbox? Secondly, that page gives instructions on how to fix the problem with mbox and maildir backends. However, my mail is stored in mdboxes. Is there a way to re-generate UIDs for mdbox mailboxes? Thanks in advance!

2 2

CPU for Dovecot
by Miloslav Hůla 28 Nov '16

28 Nov '16

Hi, we are planning to change hardware for our standalone Dovecot instance handling ~5800 IMAP users with 1TB mailboxes on local RAID. Is there some recommendation about CPU? We can choose from: - Intel Xeon E5-2620v4 - 2,1GHz@8,0GT 20MB cache, 8core, HT, 85W, LGA2011 - Intel Xeon E5-2623v4 - 2,6GHz@8,0GT 10MB cache, 4core, HT, 85W, LGA2011 The difference is about more cores vs. hi frequency. Thank you, Miloslav

4 4

Dovecot proxy
by Gandalf Corvotempesta 26 Nov '16

26 Nov '16

Hi to all I have some *production* pop3/inappropriate server that i would like to move under a proxy Some questions: 1. Keeping the same original hostname on the proxy (in example mail.mydomain.tld) and changing the hostname on the imap server, makes some troubles like MUA redownloading all the messages? Is dovecot (running on the imap server) happy seeing the hostname change? What about maildirs, where the hostname is wrote on the mail file? 2. Dovecot proxy will proxy the whole pop3/imap traffic or only the login auth? I don't want to expose the mailservers to internet, all imap session must be proxied through the proxy. this because I'll use local IPs on each mail server. 3. Performance for the proxy server? The same as the mailserver or higher due to the missing email computation? In example, the proxy doesn't have to access disks or emails data but has only to transmit what the mailserver osd saying 4. Like question 3, any real users for the proxy? I would like to know some info about hardware and userbase (in example: dual quad xeon 5600, 32Gb ram, 10.000 concurrent sessions) In my case I'm planning for about 100 active sessions. Can i use a small EC2 instance?

1 1

Re: (build) crash in dcrypt when "-Bsymbolic-functions" is used on linking
by Aki Tuomi 25 Nov '16

25 Nov '16

On 26.10.2016 13:48, Christian Ehrhardt wrote: > Hi, > I was wondering about a crash when building dovecot 2.2.25 on latest > Ubuntu. > I wondered as I've had the same source building on Debian just fine. > > Some debugging led me to this weird behavior: > Using this gdb command file called autoreportissue in my case: > break dcrypt_initialize > commands > p dcrypt_vfs > p &dcrypt_vfs > watch dcrypt_vfs > c > end > break dcrypt_set_vfs > commands > p dcrypt_vfs > p &dcrypt_vfs > c > end > r > > Running test-crypto on Debian and Ubuntu reported those two behaviours: > gdb -d /root/dovecot-2.2.25/src/ -x autoreportissue ./test-crypto > > Good: > Breakpoint 1, dcrypt_initialize (backend=0x555555587c02 "openssl", > set=0x0, error_r=0x0) at dcrypt.c:15 > 15 if (dcrypt_vfs != NULL) { > $1 = (struct dcrypt_vfs *) 0x0 > $2 = (struct dcrypt_vfs **) 0x555555796370 <dcrypt_vfs> > Hardware watchpoint 3: dcrypt_vfs > Breakpoint 2, dcrypt_set_vfs (vfs=0x7ffff7835020 <dcrypt_openssl_vfs>) > at dcrypt.c:56 > 56 dcrypt_vfs = vfs; > $3 = (struct dcrypt_vfs *) 0x0 > $4 = (struct dcrypt_vfs **) 0x555555796370 <dcrypt_vfs> > Hardware watchpoint 3: dcrypt_vfs > Old value = (struct dcrypt_vfs *) 0x0 > New value = (struct dcrypt_vfs *) 0x7ffff7835020 <dcrypt_openssl_vfs> > dcrypt_set_vfs (vfs=0x7ffff7835020 <dcrypt_openssl_vfs>) at dcrypt.c:57 > 57 } > > Bad: > Breakpoint 1, dcrypt_initialize (backend=0x555555589f02 "openssl", > set=0x0, error_r=0x0) at dcrypt.c:11 > 11 { > $1 = (struct dcrypt_vfs *) 0x0 > $2 = (struct dcrypt_vfs **) 0x555555798370 <dcrypt_vfs> > Hardware watchpoint 3: dcrypt_vfs > Breakpoint 2, dcrypt_set_vfs (vfs=0x7ffff780a020 <dcrypt_openssl_vfs>) > at dcrypt.c:56 > 56 dcrypt_vfs = vfs; > $3 = (struct dcrypt_vfs *) 0x0 > $4 = (struct dcrypt_vfs **) 0x7ffff780a890 <dcrypt_vfs> > Panic: file dcrypt.c: line 34 (dcrypt_initialize): assertion failed: > (dcrypt_vfs != NULL) > Error: Raw backtrace: > /root/dovecot-2.2.25/src/lib-dcrypt/test-crypto(+0x15f7c) > [0x555555569f7c] -> > /root/dovecot-2.2.25/src/lib-dcrypt/test-crypto(default_error_handler+0) > [0x55555556a030] -> > /root/dovecot-2.2.25/src/lib-dcrypt/test-crypto(i_fatal+0) > [0x55555556a2ff] -> > /root/dovecot-2.2.25/src/lib-dcrypt/test-crypto(dcrypt_initialize+0x140) > [0x55555555f030] -> > /root/dovecot-2.2.25/src/lib-dcrypt/test-crypto(main+0x23) > [0x55555556706d] -> > /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) > [0x7ffff782d3f1] -> > /root/dovecot-2.2.25/src/lib-dcrypt/test-crypto(_start+0x2a) > [0x55555555edea] > Program received signal SIGABRT, Aborted. > > One can see that in the bad case the pointer of dcrypt_vfs is pointing > to something of the scope of the .libs/libdcrypt_openssl.so file and > not the dcrypt_initialize of test-crypto. > > That made me wonder even more - where would this issue of variable > scope come from. After more debugging I found that the linker flag > "-Bsymbolic-functions" is the reason. This is default on recent > Ubuntu, but not on Debian (?yet?). > > Eventually what happens is that the dcrypt_vfs becomes part of the > .libs/libdcrypt_openssl.so. So the call from there to dcrypt_set_vfs > ends up setting not the expected variable. > I was unable to come up with a reasonable fix since I'm not enough > into your sublib structure. > > For now I assume I'm gonna build the package stripping this flag in > Ubuntu. > But long term I think dovecot should fix it to work with that compiler > flag. > Therefore the report to make you aware. > > > P.S. thanks to the dovecot community for having unit tests that find > this at build time! > > -- > Christian Ehrhardt > Software Engineer, Ubuntu Server > Canonical Ltd Hi! Thank you for reporting this, we'll look into it. Aki Tuomi Dovecot oy

1 1

Could the index of dovecot be stored in database?
by eSX 25 Nov '16

25 Nov '16

Hi dovecot, I have a question about the index of dovecot. I had read the wiki, and saw that I can custom a Input streams to do read/seek/sync...by implementing a istream-*.c file. Thus maybe dovecot can read mail from a http server or any others. But I found that the index of maildir can't extend very well. There is no extern API for developer? For example, I want to put the index to database(maybe, MySQL) or any else, But I found I have no way to implement it, Unless a heavy modification? -- eSX

1 0

Patch for dovecot-antispam-plugin to support IMAP Flags indicating Junk status
by Jonas Wielicki 25 Nov '16

25 Nov '16

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA512 Dear list, I made a patch which extends dovecot-antispam-plugin with support to take the addition and removal of a specific set of flags as trigger for learning. This allows for very tight integration with Thunderbird and KMail, which both set flags for Junk messages (Thunderbird sets "Junk", KMail sets "JUNK"). With this patch and according configuration, dovecot-antispam-plugin can trigger on the addition (learn as spam) and removal (learn as ham) of these flags. Combined with a sieve script which sets these flags for incoming mail which has been classified as spam, this allows for a tight and intuitive feedback loop between users and the server-side spam classification system. I tested that patch with dovecot 2.2 from Debian stretch and so far it works pretty well. I realise that dovecot-antispam-plugin has not been updated since 2013. It also has not been migrated to Github yet. If it is to be migrated there, I would be happy to submit a pull request there for discussion and merging. Otherwise, someone would have to indicate how to go forward with applying this patch to the official version, if that is desired. The patch is based on current tip and feedback is appreciated. I took some inspiration from dovecot-core/src/plugins/notify/notify-storage.c -- this requires the use of struct mail_private to be able to override the vfunc for mail_update_keywords, which I actually do not like. However, when I tried to make this patch base on the use of the notify plugin (via notify_register), I ran into the issue that the transaction is partially freed before the transaction_commit event from notify is emitted, which is unfortunate and breaks at least the mailtrain backend (which I use). This is in theory fixable, but I wanted to keep the patch as small as possible, as I have no idea whether this will eventually end up upstream or whether I’ll have to maintain that for myself. If upstream is not interested in this patch, please let me know! In that case, I would consider making an own plugin, which fewer backends (possibly only a mailtrain-like one) out of this, but frankly, I would consider that a waste of resources. In the hope that this is useful for others and hoping for feedback, Jonas Wielicki -----BEGIN PGP SIGNATURE----- iQIzBAEBCgAdFiEEG/EPV+Xzd5wEoQQIwGIDJZdiWIoFAlg4Fd0ACgkQwGIDJZdi WIpM3Q//TWVHDFx5LK14y0jOnxzE7kGthc6hakILnWqYgl7B/wlljD1HLrec//hg 9LEancjURnv3sPU/kEEpl6RRDX5kDMbXXKXMq7/1R7bNdmeMz5NEebAo7ktlCii7 ikk7MPL0F8UViY7IoW7y5NxfqMUldYseejn0GwKiVkEyqlyGLRbP7ijyzMWjpIWp eqOg/b5bNX6ARMC1O/hKhvHw4AhvSO2j73da60Xchu39gvDa2kC+MTU/kX4wb6po kPOEgFxGhpxG0+UMzUEBPLYwY3Q9x+R/PzlpUF6xEjYxgE7leVRVfCHiZLsMUGH1 itozwSzoAyDYV+Rbzk8OEBL6EGBJQHqYmNybULZsJkxhUenNeC4GRf7NcJ0jAisG rSfwr/+kD9+ErKNgtNJzfceLYGNWti44whV6kXxRxqGlIfJsJB7ZMdrj8+wSdn0p slXUoyHMUkRkN92tIPqZF5cb+5Ke+O+5hHE07Wohxw3eNoBSqtQWo81aigjaCraC ivcZX3HswTapyKtpgWMzjoWqp982crr+wtAI+AlEUKoCEqF54EnxM1kB1qoe0HcH nMgMdIkXI4VzPaUNUK206ffEtlnKNIqF3SkOkBX263L0ZJgXyCFRRj6W3z78VDdP He62tN/FPPBtmeVrmCj2ZDE+6gqtMQN52FsN4mpaq4BrCYlYB7g= =3+CU -----END PGP SIGNATURE-----

1 0

Updated my Dovecot certificate for the first time
by Steve Litt 25 Nov '16

25 Nov '16

Hi all, I've used Dovecot since February 2012, but because I kept reinstalling Linux with every major version, I never had a Dovecot self-signed certificate go bad on me before. Til now. I started using rolling release Void Linux about a year ago, and my Dovecot self-signed certificate just expired. The solution I used is contained in these documents: http://wiki2.dovecot.org/SSL/CertificateCreation http://wiki2.dovecot.org/SSL/CertificateClientImporting file:///etc/ssl/dovecot-openssl.cnf http://www.faqforge.com/linux/renew-the-dovecot-ssl-certificate-on-ubuntu-l… I basically moved my old /etc/ssl/certs/dovecot.pem and /etc/ssl/private/dovecot.pem, then edited /etc/ssl/dovecot-openssl.cnf specifically to give myself the common name of 192.168.100.2. I had earlier used my hostname, but that produced a conflict, so I just used the ip address. Then I ran dovecot-mkcert.sh to create the new self-signed cert, and finally, configured Claws-Mail to use /etc/ssl/certs/dovecot.pem as its cert. Obviously, if my Claws-Mail were on a different machine than my Dovecot, I would have had Claws-Mail point to a local copy. Alpine still gives me a bad cert warning, saying I should either fix it or disable checking. I haven't yet found a way to get Alpine to discriminate between a valid self-signed cert and a bad one. Anyway, all's good. SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz

7 9

Re: Updated my Dovecot certificate for the first time
by Joseph Tam 24 Nov '16

24 Nov '16

On Thu, 24 Nov 2016, Steve Litt wrote >> add the public part of the cert into your system's trusted CA store. > > Silly question, but how would you do that? You didn't say which OS you're running on (alpine runs on Windows as well), but I'll assume *nix. A previous poster showed you how to do it with a real certificate, and the steps are the same. However, the way I found out without too much fuss was to process trace my alpine process and see where it tied to load a cert $ strace -o trace.out alpine ... quit after connection $ grep -F cert traceout /1: open64("/etc/openssl/cert.pem", O_RDONLY) Err#2 ENOENT /1: stat("/etc/openssl/certs/cbf06781.0", 0xFFBF8E54) Err#2 ENOENT ... Your output will be different of course. The first load is the default pre-loaded root CAs (Thawte, etc.) supplied by OpenSSL, and the second, etc. are chained certificate lookups. You would replace the missing cert with your own self-signed public pem file. e.g. cp mypub.pem /etc/openssl/certs/cbf06781.0 For Windows, I don't know where it fetches it from. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: Updated my Dovecot certificate for the first time
by Joseph Tam 24 Nov '16

24 Nov '16

> Alpine still gives me a bad cert warning, saying I should either fix it > or disable checking. I haven't yet found a way to get Alpine to > discriminate between a valid self-signed cert and a bad one. Well, it can't discriminate since any certificate (except those in your trusted store) that asserts its own validity is suspect. You can either get alpine to not complain e.g. inbox-path={192.168.100.2:143/user=whatever/tls/novalidate-cert} or add the public part of the cert into your system's trusted CA store. Joseph Tam <jtam.home(a)gmail.com>

2 1

antispam plugin not working correctly
by Felix Zielcke 23 Nov '16

23 Nov '16

Hi, I've configured the antispam plugin like in the wiki to use the sa- learn-pipe.sh shell script. But it's not creating the log file in /tmp. Even if I add a "touch /tmp/x" in sa-learn-pipe.sh it's not created. So how I can check if it works correctly? dovecot version 2.2.26.0 /var/log/debug although contains: Nov 23 18:36:49 daedalus imap [fzielcke(a)z-51.de 80.187.96.201 UID MOVE]: antispam: mail copy: src spam: 1, dst spam: 0, src unsure: 0 Nov 23 18:36:49 daedalus imap [fzielcke(a)z-51.de 80.187.96.201 UID MOVE]: antispam: running mailtrain backend program /usr/bin/sa-learn-pipe.sh Nov 23 18:36:49 daedalus imap [fzielcke(a)z-51.de 80.187.96.201 UID MOVE]: antispam: running mailtrain backend program /usr/bin/sa-learn-pipe.sh Nov 23 18:36:49 daedalus imap [fzielcke(a)z-51.de 80.187.96.201 UID MOVE]: antispam: running mailtrain backend program parameter 1 --ham # doveconf -n |grep antispam antispam_backend = pipe antispam_debug_target = syslog antispam_pipe_program = /usr/bin/sa-learn-pipe.sh antispam_pipe_program_notspam_arg = --ham antispam_pipe_program_spam_arg = --spam antispam_pipe_tmpdir = /tmp antispam_spam_pattern_ignorecase = SPAM antispam_trash_pattern_ignorecase = trash;Deleted * antispam_verbose_debug = 1 mail_plugins = acl zlib quota imap_zlib imap_acl imap_quota antispam

1 0

Active sieve script symbolic link doesn't update when changed.
by Steven Mainor 23 Nov '16

23 Nov '16

While setting up sieve on my email server I mistakenly setup a symlink (~/sieve/myfilter.svbin > ~/.dovecot.sieve) but I think that that should have been (~/sieve/myfilter.sieve > ~/.dovecot.sieve). I have fixed the mistake but dovecot/sieve still throws an error saying that the symlink still links to the wrong file. I have made sure the symlink is fixed (it links to ~/sieve/myfilter.sieve) and I tried restarting dovecot and the server but nothing works. I still get the error and my sieve filters aren't processed. Any help would be greatly appreciated. ~# grep sieve /var/log/syslog | nano temp.save Nov 23 01:32:09 1 dovecot: lda(steve(a)degga.net): Warning: sieve: file storage: Active Sieve script symlink /var/mail/vmail/degga.net/steve/.dovecot.sieve is broken: Invalid scriptname (points to myfilter.svbin). vmail@1:/var/mail/vmail/degga.net/steve/mail$ ls -l -a #redacted irrelevant stuff lrwxrwxrwx 1 vmail vmail 14 Nov 23 02:43 .dovecot.sieve -> myfilter.sieve vmail@1:/var/mail/vmail$ dovecot -n # 2.2.22 (fe789d2): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.13 (7b14904) # OS: Linux 4.4.0-47-generic x86_64 Ubuntu 16.04.1 LTS ext4 auth_mechanisms = plain login auth_verbose = yes mail_home =/var/mail/vmail/%d/%n mail_location = maildir:/var/mail/vmail/%d/%n/mail:LAYOUT=fs managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes location = mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = } passdb { args = username_format=%u scheme=ssha512 /etc/dovecot/passwd.db driver = passwd-file } plugin { sieve = ~/.dovecot.sieve sieve_after = /var/mail/vmail/sieve-after sieve_before = /var/mail/vmail/sieve-before sieve_dir = ~/sieve } protocols = imap sieve service auth { unix_listener /var/spool/postfix/private/dovecot-auth { group = postfix mode = 0660 user = postfix } } # SSL information redacted here. userdb { args = uid=vmail gid=vmail home=/var/mail/vmail/%d/%n driver = static } protocol lda { deliver_log_format = msgid=%m: %$ mail_plugins = sieve postmaster_address =postmaster(a)degga.net quota_full_tempfail = yes rejection_reason = Your message to <%t> was automatically rejected:%n% r } protocol imap { imap_client_workarounds = delay-newmail tb-extra-mailbox-sep mail_max_userip_connections = 10 }

2 2

Exim still accepting emails to nonexistent users
by Marti Markov 23 Nov '16

23 Nov '16

After adding the configuration bit: deny message = invalid recipient domains = +local_domains !verify = recipient/callout=no_cache from: http://wiki2.dovecot.org/LMTP/Exim <http://wiki2.dovecot.org/LMTP/Exim> running update-exim4.conf and service exim4 restart the server is still accepting emails to recipients that do not exist in dovecot. Any ideas?

3 6

Single user to have lrw to all mailboxes
by Marti Markov 23 Nov '16

23 Nov '16

I have implemented a global acl: * owner lrwstipkae INBOX.Spam owner lrwstipeka .Trash owner lrwsti INBOX.Trash owner lrw * user=master-user(a)mydomain.com lrs * owner lrwstipkae But the master-user then loses all other permissions on his own mailbox, he is left only with: lrs Any pointers on how to fix this?

1 0

Good email client to use with Dovecot?
by Steve Litt 22 Nov '16

22 Nov '16

Hi all, When I use an email client, its purpose is as a window into my Dovecot IMAP, and as a mechanism to reply to and send emails. I don't do filtering or calendaring on my email client (filtering via procmail direct to Dovecot). What email clients are all of you using to look at your IMAP email? Thanks, SteveT Steve Litt November 2016 featured book: Quit Joblessness: Start Your Own Business http://www.troubleshooters.com/startbiz

20 40

LDAP auth problems "unknown user"
by Nikolai Lusan 22 Nov '16

22 Nov '16

Hi, I am setting up a dovecot instance to host a bunch of virtual domains, with ldap backing for auth. I am using a single hostname for test purposes, with a couple of entries in the diectory. If I use auth binds I get a result, but this does not suit the end purpose of the server. If I use ldapsearch with the same base, search filter, and credentials as the ldap auth config I get the correct attributes returned, and I have bit of perl that verifies the password hash matches the password provided. I am running out of ideas here, I had thought of putting in a custom bit of perl and using the checkpassword method, but this is sub optimal. If anyone can help with this I'd be grateful. # dovecot --version 2.2.26.0 (23d1de6) # dovecot -n # 2.2.26.0 (23d1de6): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.8.0-1-amd64 x86_64 Debian stretch/sid ext3 auth_cache_size = 2 M auth_debug = yes auth_debug_passwords = yes auth_default_realm = maliuta.org auth_master_user_separator = * auth_mechanisms = plain login auth_realms = maliuta.org auth_verbose = yes auth_verbose_passwords = yes first_valid_uid = 117 last_valid_uid = 117 lda_mailbox_autocreate = yes login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_gid = vmail mail_location = maildir:/var/spool/vmail/%d/%n/Maildir mail_uid = vmail managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded- character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes list = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /etc/dovecot/ldap/maliuta.org-ldap.conf.ext driver = ldap } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = " imap lmtp sieve" service auth-worker { user = $default_internal_user } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0660 user = vmail } user = $default_internal_user } service dict { unix_listener dict { group = vmail mode = 0660 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 0 } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } ssl_cert = # grep -v '^ *$#.*$\?$' /etc/dovecot/ldap/maliuta.org-ldap.conf.ext uris = ldap://localhost dn = cn=admin,dc=maliuta,dc=org dnpass = <secret> tls = yes tls_ca_cert_dir = /etc/ssl/certs auth_bind = no ldap_version = 3 base = ou=mail,dc=mailuta,dc=org scope = subtree default_pass_scheme = SSHA deref = never user_attrs = postfixDeliveryAddress=user user_filter = (&(postfixDeliveryEnabled=TRUE)(objectClass=postfixMailPerson)) pass_attrs = postfixDeliveryAddress=user,userPassword=password pass_filter = (&(postfixDeliveryEnabled=TRUE)(objectClass=postfixMailPerson)(postfixD eliveryAddress=%u)) iterate_attrs = uid=user iterate_filter = (objectClass=postfixMailPerson) # ldapsearch -H ldap://localhost:389 -x -D 'cn=admin,dc=maliuta,dc=org' -W -b "ou=mail,dc=maliuta,dc=org" -s sub -LLL -ZZ '(&(postfixDeliveryEnabled=TRUE)(objectClass=postfixMailPerson)(postfixDeliveryAddress=nikolai(a)test.maliuta.org))' uid userPassword Enter LDAP Password: dn: mail=nikolai(a)test.maliuta.org,ou=mail,dc=maliuta,dc=org uid: nikolai userPassword:: e1NTSEF9QVBZMTlaeGw1cWd0a25XeGxURXdqM2g5Yk5YL3BxOGY= ## From /var/log/mail.log Nov 20 07:24:20 kiliya dovecot: auth: Debug: auth client connected (pid=27086) Nov 20 07:24:20 kiliya dovecot: auth: Debug: client in: AUTH#0111#011PLAIN#011service=imap#011secured#011session=1kW2C65BFI2WZbl8#011lip=<local_ip>#011rip=<remote_ip>#011lport=143#011rport=36116#011local_name=<hostname>#011resp=AG5pa29sYWlAdGVzdC5tYWxpdXRhLm9yZwBmb29iYXIzMzQ0 (previous base64 data may contain sensitive data) Nov 20 07:24:20 kiliya dovecot: auth: Debug: ldap(nikolai(a)test.maliuta.org,<remote_ip>,<1kW2C65BFI2WZbl8>): cache miss Nov 20 07:24:20 kiliya dovecot: auth: Debug: ldap(nikolai(a)test.maliuta.org,<remote_ip,<1kW2C65BFI2WZbl8>): pass search: base=ou=mail,dc=mailuta,dc=org scope=subtree filter=(&(postfixDeliveryEnabled=TRUE)(objectClass=postfixMailPerson)(postfixDeliveryAddress=nikolai(a)test.maliuta.org)) fields=postfixDeliveryAddress,userPassword Nov 20 07:24:20 kiliya dovecot: auth: ldap(nikolai(a)test.maliuta.org,<remote_ip>,<1kW2C65BFI2WZbl8>): unknown user (given password: <correct_password>) Nov 20 07:24:22 kiliya dovecot: auth: Debug: client passdb out: FAIL#0111#011user=nikolai(a)test.maliuta.org -- Nikolai Lusan <nikolai(a)lusan.id.au>

2 2

Maildir migration from Dovecot to Dovecot
by Marco Hofmann (XenAdmin) 22 Nov '16

22 Nov '16

Hello everbody, I'm a proud user of Dovecot 2.2.9 on Ubuntu 14.04 LTS. I currently migrate all my Linux services to a new Debian 8 Jessie Server. On this server I installed and configured Dovecot 2.2.24. Now I search a way to migrate the two IMAP users I have on this server preserving all timestamps, folders and flags. Yes, only two ;-) I read the following two articles: http://wiki2.dovecot.org/Tools/Doveadm/Sync http://wiki2.dovecot.org/Migration/Dsync But I must say, that either these articles are not so good, or I act quite dumb. I understand that I have to use "doveadm backup" against my old server to make a one sync with all properties of each mail. What I miss are proper examples and that there are so many options, I could set. But what confuses me most is the the configuration file in (http://wiki2.dovecot.org/Migration/Dsync) which I don't know how to reference? Am I using the wrong guides, are the article someway misleading or is there anything else I'm overlooking? Thank you so much! -- Marco Hofmann https://www.meinekleinefarm.net/ Twitter: @ZomboBrain

2 1

nologin + reason -> logging reason
by Arkadiusz Miśkiewicz 22 Nov '16

22 Nov '16

Hi. I'm using nologin with own reason [1]. That works fine. For example pop3 client gets nice message like "-ERR [AUTH] Account is locked. Please contact support." Unfortunately maillog lacks information details about why user was not allowed to log in. pop3-login: Disconnected (auth failed, 1 attempts in 2 secs): user=<testuser>, method=LOGIN, rip=1.1.1.1, lip=2.2.2.2, session=<yIsRftBBmsrB7y2h> Is it possible to log "reason" there, too? (whether it is default/internal dovecot reason or my custom one). reason is the only thing in maillog that allows to distinguish why user was not allowed to log in. 1. http://wiki2.dovecot.org/PasswordDatabase/ExtraFields -- Arkadiusz Miśkiewicz, arekm / ( maven.pl | pld-linux.org )

3 5

Clients creating folders in root when using shared namespace
by Jon Cutting 21 Nov '16

21 Nov '16

We have set up dovecot to use a shared namespace, and it works well. However, when a client attempts to create a folder in the namespace, although dovecot responds with permission denied, it still goes on to create a folder, but in the root of our virtual mailbox folder (more confusingly, even issuing a LIST command for a non-existent subfolder creates the folder in the root). I've checked through the documentation for a misconfiguration but so far to no avail. Our steps to reproduce are as follows: * Create shared namespace (in our case we've used prefix Users/%%u/) * Share folder from one user to another so that the shared folder displays in LIST * Create a folder as a subfolder of the shared folder (i.e. . CREATE Users/testfolder) Expected behaviour: permission is denied and no folder is created Actual behaviour: permission is denied, but folder is created at the root of our vmail store (in the above example a folder /mnt/storage/vmail/testfolder would be created) Thanks, Jon Output from doveconf -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 ext4 mail_location = maildir:/mnt/storage/vmail/%d/%n/Maildir mail_plugins = " quota zlib acl" managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace Users { list = children location = maildir:/mnt/storage/vmail/%%d/%%n/Maildir:INDEX=/mnt/storage/vmail/%d/%n/sharedindex/%%d/%%n/ prefix = Users/%%u/ separator = / subscriptions = no type = shared } namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = / type = private } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { acl = vfile acl_shared_dict = file:/mnt/storage/vmail/%d/shared-mailboxes.db quota = maildir:User quota quota_rule = *:storage=50GB sieve = /mnt/storage/vmail/%d/%n/.dovecot.sieve sieve_before = /mnt/storage/vmail/dovecot.sieve sieve_dir = /mnt/storage/vmail/%d/%n/sieve zlib_save = gz zlib_save_level = 6 } protocols = imap lmtp sieve service imap-login { inet_listener imaps { port = 993 ssl = yes } } service lmtp { inet_listener lmtp { address = 0.0.0.0 port = 2003 } user = vmail } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 vsz_limit = 64 M } ssl = required ssl_ca = </etc/dovecot/dovecotca.pem ssl_cert = </etc/dovecot/dovecot.pem ssl_key = </etc/dovecot/private/dovecot.pem userdb { args = uid=5000 gid=5000 home=/mnt/storage/vmail/%d/%n allow_all_users=yes driver = static } protocol imap { imap_client_workarounds = tb-extra-mailbox-sep mail_plugins = " quota zlib acl imap_quota" } protocol lmtp { mail_plugins = " quota zlib acl sieve" }

1 0

[PATCH] Manually cleanup OpenSSL from dovecot_openssl_common_global_unref()
by Apollon Oikonomopoulos 21 Nov '16

21 Nov '16

OpenSSL 1.1 features a cleanup function that is automatically run on shutdown using atexit(3). This function frees all OpenSSL-allocated resources. In dovecot, OpenSSL is loaded indirectly using dlopen(3) against the relevant dovecot crypto module and is finally unloaded using dlclose(3). Until OpenSSL 1.0.1c this worked fine, however OpenSSL 1.0.1c makes sure[1] that the library stays loaded after the initial dlclose() so that the atexit(3) handlers can run on shutdown. This, together with the fact that dovecot uses custom allocation functions for OpenSSL and has already partially free()'d some of OpenSSL's resources in module_free(), leads to a segfault at process shutdown[2]. We fix this by explicitly calling OPENSSL_cleanup() during module unload. This is safe to do, as long as we will never want to subsequently re-initialize OpenSSL. [1] https://github.com/openssl/openssl/commit/4af9f7fe79ff82b90c16969b7e5871435… [2] https://buildd.debian.org/status/fetch.php?pkg=dovecot&arch=amd64&ver=1:2.2… Signed-off-by: Apollon Oikonomopoulos <apoikos(a)debian.org> --- src/lib-ssl-iostream/dovecot-openssl-common.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/src/lib-ssl-iostream/dovecot-openssl-common.c b/src/lib-ssl-iostream/dovecot-openssl-common.c index 51ea3ad..2bf6307 100644 --- a/src/lib-ssl-iostream/dovecot-openssl-common.c +++ b/src/lib-ssl-iostream/dovecot-openssl-common.c @@ -101,6 +101,9 @@ bool dovecot_openssl_common_global_unref(void) ERR_remove_thread_state(NULL); #endif ERR_free_strings(); +#if OPENSSL_VERSION_NUMBER >= 0x10100000L + OPENSSL_cleanup(); +#endif return FALSE; } -- 2.10.1

4 4

Index-cache sizes
by Tom Sommer 21 Nov '16

21 Nov '16

I had a customer with an INBOX cache of 400MB+ (5.5GB maildir) which gave me "Error: mmap_anon(474165248) failed: Cannot allocate memory" errors, then I deleted the cache files and ran 'doveadm index -u xxx(a)example.com "*"', now the INBOX cache is 7MB What is up with that? Is the cache now missing something or does it make sense to wipe cache files and reindex cache every-so-often? -- Tom

1 0

Feedback on first, i.e. novice-level, experiences with dovecot
by Michael Felt 21 Nov '16

21 Nov '16

a) google: "dovecot imap configuration" brought an article from 2005 as the first item: while I like debian and a bit of history you may want to ask yourself how to improve your site as the primary hit from google. FYI: years ago I had great results in the top of google searches - and then I made a major error - I moved my site to a new url and google forgot me. b) using your http://wiki.dovecot.org/QuickConfiguration as a guide: I built dovecot using --prefix=/opt (I abhor /usr/local these days - that is so 1984ish). The applications got installed in /opt/bin root@x066:/data/prj/aixtools/dovecot/core# ls -ltr /opt total 36 drwxr-xr-x 3 root root 4096 Nov 20 14:33 lib drwxr-xr-x 3 root root 4096 Nov 20 14:33 include drwxr-xr-x 6 root root 4096 Nov 20 14:33 share drwxr-xr-x 3 root root 4096 Nov 20 14:33 libexec drwxr-xr-x 2 root root 4096 Nov 20 14:33 sbin drwxr-xr-x 2 root root 4096 Nov 20 14:33 bin Are the directories 'make install' touched or made. root@x066:/data/prj/aixtools/dovecot/core# /opt/bin/doveconf -n # 2.3.devel (92c8109): /opt/etc/dovecot/dovecot.conf doveconf: Fatal: open(/opt/etc/dovecot/dovecot.conf) failed: No such file or directory where are example configs? root@x066:/data/prj/aixtools/dovecot/core# find /opt -name example-config root@x066:/data/prj/aixtools/dovecot/core# Not in /opt How about a default config? root@x066:/data/prj/aixtools/dovecot/core# find / -name dovecot /etc/init.d/dovecot /etc/pam.d/dovecot /etc/default/dovecot /etc/dovecot /var/lib/dovecot ^C Seems to be /etc/dovecot root@x066:/data/prj/aixtools/dovecot/core# ls -l /etc/dovecot total 40 drwxr-xr-x 2 root root 4096 Apr 30 2015 conf.d -rw-r--r-- 1 root root 4180 Jun 8 2014 dovecot.conf -rw-r----- 1 root dovecot 410 Jun 8 2014 dovecot-db.conf.ext -rw-r----- 1 root dovecot 782 Jun 8 2014 dovecot-dict-sql.conf.ext -rw-r--r-- 1 root dovecot 1363 Apr 30 2015 dovecot.pem -rw-r----- 1 root dovecot 5348 Jun 8 2014 dovecot-sql.conf.ext drwx------ 2 root root 4096 Apr 30 2015 private -rw-r--r-- 1 root root 121 Jun 8 2014 README and finally - read README to find the examples: root@x066:/data/prj/aixtools/dovecot/core# cat /etc/dovecot/README Configuration files go to this directory. See example configuration files in /usr/share/doc/dovecot-core/example-config/ Conclusion: --prefix=/opt is not applied everywhere. * More first impressions as I come to them - if you want them - as I hope this helps make dovecot better! Michael

2 4

doveadm service: verify client cert
by Matwey V. Kornilov 20 Nov '16

20 Nov '16

Hello, I want to open a socket for connecting doveadm using tcps. I do the following: service doveadm { inet_listener { port = 5001 ssl = yes } } At the same time, I would like to verify client certificates for connections goes to port 5001. I am trying to do the following, but it doesn't work: protocol doveadm { ssl_require_crl = yes ssl_verify_client_cert = yes } How could I achieve required behavior?

1 0

Compound Literal - xlc and gcc differences can be patched
by Michael Felt 19 Nov '16

19 Nov '16

Since I so miserably misspelled packaging - a new thread specific to the issue at hand. I found a "workaround". In short, xlc does not accept arrays of nested array - the bottom line (best message) is: 1506-995 (S) An aggregate containing a flexible array member cannot be used as a member of a structure or as an array element. At the core - all the other messages come because this (I shall call it nesting). For the test-http-auth.c file I have a patch (attached). make ran for while with no issue, then it started running into several files with this same style of specification - doveadm-dict.c being the first one. I ran "make -i" and I notice several files with this - for xlc - "unaccepted" syntax. Please look at the patch - what it basically is - and let me know whether you would consider accepting patches in this form (e.g., verify gcc (I assume) will accept it as well). If your answer is yes, I shall proceed with the additional files (learn how your MACROS work!) and send them to you. Again, initially I would just send one, e.g., doveadm-dict.c - to be sure your regular compiler also builds this alternate specification. Thank you for your consideration! Michael

2 5

Re: Good email client to use with Dovecot?
by Joseph Tam 19 Nov '16

19 Nov '16

Steve Litt asks > * And most important, where can I find the best, most unambiguous > Alpine documentation, that doesn't assume I know anything about > Alpine? The ~/.pinerc config file is well documented. There is a bewildering number of options, and you can tweak it just about anyway you like. Choice of mail readers tend to a religious issue. Users of a particular reader will only change when you pry their cold dead fingers off of it. Alpine is not a bad choice (I've been using it and it's predeccesor for decades), and I go through mail faster than most people I know as I can type faster than they can click buttons, scroll, and drag windows around. However, it sort of sucks when you read formatted mail HTML with lots of graphics). Joseph Tam <jtam.home(a)gmail.com>

1 0

Generalized looping possible in pigeonhole sieve?
by Jeff Kletsky 18 Nov '16

18 Nov '16

I'm struggling to achieve the kind of filtering with sieve that I was able to do with procmail. TL;DR I'd like a way to loop through a set of (address, destination) pairs in sieve so that I can maintain the (address, destination) pairs in one place and not have to explicitly write scores of nearly identical [...] if address :matches ["From", "Sender", "To", "Cc"] "address53" { fileinto "destination53" } if address :matches ["From", "Sender", "To", "Cc"] "address54" { fileinto "destination54" } [...] Longer: I've been using a table-based dispatch approach with procmail for years and it was working reasonably well. The approach uses list of pairs of address and destination boxes and doing a lookup based on the message to be delivered. With procmail, I collected the list of addresses and passed them to external scripts to do the lookup. If I just had a dozen boxes that I deal with, that would be straightforward to implement and maintain in sieve. As the list is an order of magnitude greater than that, maintaining the list of pairs in an external file and programmatically looping through it is a lot easier. I've looked at vnd.dovecot.execute and it certainly can return a destination box name given the input information. Unfortunately, the construct of (for example) address :matches "To" "*"' only returns the first address, not the list of addresses. I can't see a good way to get all the addressees from sieve to pass to the external program. If I have to pass the message (or at least the interesting headers) to the external program and then parse the address list myself, then I'm nearly all the way to just using a full-on external program to parse the whole message. While I'm not adverse to writing the whole thing in Python (or the like) and just using sieve as a thin shell, I'd like to make sure I'm not missing something in sieve. One way to achieve this would be to be able to loop through the pairs and using variable substitution for each iteration. I haven't seen anything in Dovecot Pigeonhole sieve that allows this. If you have any ideas on how to harness sieve for this, I'd appreciate it! Jeff

3 2