- dovecot - dovecot.org

Benchmarking auth
by micah anderson 25 Jan '17

25 Jan '17

Hello, I'm attempting to use imaptest to test auths/sec on my imap server, to simulate the number I'm getting now on a new server. Based on my readings of dovecot stats outputs, my current machine is doing somewhere between 11.6 and 196 logins/sec at its busiest, and I want to make sure the new server can handle that amount. With imaptest I attempt to simulate this by just simulating login/logout speed, but I'm not sure I understand the results and where things are being blocked. For testing purposes, I set: mail_max_userip_connections = 6000 mail_max_userip_connections = 6000 auth_worker_max_count = 240 If you have some additional suggestions for what I can also tweak here to improve this, I would like to know. I then ran imaptest with: imaptest clients=196 user=test pass=testpw host=127.0.0.1 port=14300 - select=0 seed=123 secs=300 It doesn't seem like I can simulate *only* authentications/logins. I have to pair them with logouts. It makes me wonder how I can simulate the amount of load on my current live system on this system. How do I read the output? I don't really understand what is being shown here: Logi Logo 100% 100% 0 0 100/100 [51%] does this mean that it tried 100 logins and 100 logouts and 51% succeeded? What are the first two columns? 7 7 104/121 [61%] 6 6 131/139 [70%] 3 3 137/148 (84 stalled >3s) [75%] 84 stalled for more than 3 seconds, how does this relate to the other values presented? 5 5 155/163 (79 stalled >3s) [83%] 1 1 166/166 (107 stalled >3s) [84%] 2 2 168/172 (128 stalled >3s) [87%] 6 5 179/190 (140 stalled >3s) [96%] 22 22 182/196 (136 stalled >3s) 4 3 195/196 (133 stalled >3s) 5845 129 ms/cmd avg I really wonder if these first two columns are ms/cmds? It then continues: Logi Logo 100% 100% 10 11 195/196 (131 stalled >3s) 7 8 194/196 (150 stalled >3s) 3 3 194/196 (171 stalled >3s) 10 9 192/196 (166 stalled >3s) 22 22 182/196 (156 stalled >3s) 8 9 193/196 (153 stalled >3s) - 2 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 7 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 8 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 9 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 11 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 12 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 14 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 22 stalled for 16 secs in command: 1 LOGIN "test" "testpw" - 25 stalled for 16 secs in command: 1 LOGIN "test" "testpw" ... it starts to build up like this, until I hit control-c twice: - 100 stalled for 17 secs in command: 1 LOGIN "test" "testpw" ^CInfo: Received second SIGINT - stopping immediately 11347 118 ms/cmd avg Totals: Logi Logo 100% 100% 130 134 What are these totals? Thanks! micah

1 0

Apparent Maildir permission issue
by Mark Foley 25 Jan '17

25 Jan '17

I've just upgraded from Slackware 14.1 to 14.2. I've not done anything with dovecot -- it's the same version that was running before the upgrade. However, now I'm getting a permission error: /var/log/maillog: Jan 16 13:09:44 mail dovecot: imap(mark): Error: opendir(/home/HPRS/mark/Maildir) failed: Permission denied (euid=3000026(HPRS\mark) egid=100(users) missing +r perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail last message repeated 4 times Jan 16 13:09:44 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/.Trash) failed: Permission denied (euid=3000026(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail dovecot: imap(mark): Error: stat(/home/HPRS/mark/Maildir/tmp) failed: Permission denied (euid=3000026(HPRS\mark) egid=100(users) missing +x perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:44 mail dovecot: imap(mark): Error: opendir(/home/HPRS/mark/Maildir) failed: Permission denied (euid=3000026(HPRS\mark) egid=100(users) missing +r perm: /home/HPRS/mark/Maildir, conflicting dir uid=10001(HPRS\mark)) Jan 16 13:09:45 mail last message repeated 11 times Permission on that folder are: $ ls -ld /home/HPRS/mark/Maildir drwx------ 17 HPRS\mark domusers 4096 Dec 7 23:07 /home/HPRS/mark/Maildir/ Permissions are unchanged since before the backup. What do I do to fix this? THX --Mark

5 9

Timing information for passdb/userdb lookups
by Marcus Rückert 25 Jan '17

25 Jan '17

Hi, it would be nice to have an option to enable logging for timings without having to go with auth_verbose/auth_debug. If you try to track down if a slowdown for logging in comes from e.g. running out of login workers or if your auth backend is slow it would really help to get just a one line message per userdb/passdb with the timing for this lookup. something like: imap-login(<username>): userdb ldap lookup time 0.4s imap-login(<username>): passdb ldap lookup time 0.2s or even merging into one: imap-login(<username>): userdb ldap lookup time 0.4s passdb ldap lookup time 0.2s darix -- openSUSE - SUSE Linux is my linux openSUSE is good for you www.opensuse.org

1 0

Clamping down on mailbox sizes...
by SH Development 25 Jan '17

25 Jan '17

I don’t want to do this for all users….but… I have a few users who insist that they use their mailboxes regularly and don’t want it cancelled. Fine. But they won’t clean them out either. What steps would you all recommend for setting quotas on some users but not others? Specifically starting out with identifying WHICH accounts have excessive amounts of crap in them, by age, then sending them a notice stating they are going to get limited, then deleting mail older than x number of days if they don’t do it themselves by a certain time frame... Thoughts? Jeff

4 4

Re: Log authentication attempts
by Joseph Tam 25 Jan '17

25 Jan '17

On 24.01.2017 00:06, rej ex wrote: > Because we are building some monitoring application, we will need to > record all failed and successful login attempts. We need to record > remote IP, entered password in plain text, and if possible whether auth > request is for SMTP or IMAP session. SMTP? Wouldn't that be handled by your MTA, not Dovecot? AKi Tuomi wrote: > Since 2.2.27 we've had auth policy server support which can do this > properly. As I read the docs, the auth policy server would only get the hashed password, and wouldn't be able to record the plaintext password. Maybe use the checkpassword hook? http://wiki.dovecot.org/AuthDatabase/CheckPassword Joseph Tam <jtam.home(a)gmail.com>

2 1

Quota count does not work with lock_method=dotlock
by mkliewe＠gmx.de 24 Jan '17

24 Jan '17

Hello, dovecot crashes when I switch the quota tracking from dict to count. The following is working since years: quota = dict:User quota::proxy::quota I'm using a mysql quota dictionary. Now I want to switch to the new "quota count" + "quota clone" combination. I'm loading the "quota_clone" plugin, and: #quota = dict:User quota::proxy::quota quota = count:User quota quota_vsizes = yes quota_clone_dict = proxy::quota It works on a maschine where I'm using NFS with lock_method=fcntl. It's crashing on another maschine where I'm using NFS with lock_method=dotlock. See below. Sadly I have to use dotlock, because of the NFS server. It looks like quota count does not (yet) work with dotlock? Michael Jan 17 18:46:04 XXXX dovecot: imap-login: Login: user=<XXXX>, method=PLAIN, rip=10.0.XX.XX, lip=10.0.XX.XX, mpid=19123, secured, session=<XXXX> Jan 17 18:46:04 XXXX dovecot: imap(XXXX): Panic: file file-lock.c: line 285: unreached Jan 17 18:46:04 XXXX dovecot: imap(XXXX): Error: Raw backtrace: /usr/local/lib/dovecot/libdovecot.so.0(+0x93ea2) [0x7f33f7812ea2] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x93f8d) [0x7f33f7812f8d] -> /usr/local/lib/dovecot/libdovecot.so.0(i_fatal+0) [0x7f33f77ac7c1] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x98810) [0x7f33f7817810] -> /usr/local/lib/dovecot/libdovecot.so.0(file_wait_lock_error+0x26) [0x7f33f7817be6] -> /usr/local/lib/dovecot/libdovecot.so.0(file_try_lock_error+0x11) [0x7f33f7817c61] -> /usr/local/lib/dovecot/libdovecot.so.0(file_create_locked+0x17d) [0x7f33f7814d1d] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xb25f3) [0x7f33f7b4d5f3] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mailbox_get_virtual_size+0x4c) [0x7f33f7b4d91c] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(index_mailbox_get_metadata+0x1ab) [0x7f33f7b5510b] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0x5e357) [0x7f33f7af9357] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(+0xa15ac) [0x7f33f7b3c5ac] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mailbox_get_metadata+0x60) [0x7f33f7ae0d80] -> /usr/local/lib/dovecot/lib10_quota_plugin.so(quota_count+0xfe) [0x7f33f6fa6bce] -> /usr/local/lib/dovecot/lib10_quota_plugin.so(+0x7d3e) [0x7f33f6fa6d3e] -> /usr/local/lib/dovecot/lib10_quota_plugin.so(quota_get_resource+0x78) [0x7f33f6fa5578] -> /usr/local/lib/dovecot/lib10_quota_plugin.so(+0x68b4) [0x7f33f6fa58b4] -> /usr/local/lib/dovecot/lib10_quota_plugin.so(quota_over_flag_check_startup+0x78) [0x7f33f6fa63f8] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(hook_mail_namespaces_created+0x5e) [0x7f33f7ae21ae] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_namespaces_init_finish+0x46f) [0x7f33f7ad745f] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_namespaces_init+0x104) [0x7f33f7ad7674] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_next+0x6c5) [0x7f33f7ae5275] -> /usr/local/lib/dovecot/libdovecot-storage.so.0(mail_storage_service_lookup_next+0x4c) [0x7f33f7ae55dc] -> dovecot/imap(client_create_from_input+0x46) [0x425476] -> dovecot/imap() [0x42567f] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x316d3) [0x7f33f77b06d3] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x319bd) [0x7f33f77b09bd] -> /usr/local/lib/dovecot/libdovecot.so.0(+0x3235a) [0x7f33f77b135a] Jan 17 18:46:04 zzz-dovecot01 dovecot: imap(XXXX): Fatal: master: service(imap): child 19123 killed with signal 6 (core dumped)

3 5

Log authentication attempts
by rej ex 24 Jan '17

24 Jan '17

Hi everyone, We are running Dovecot 2.2.9 as a primary IMAP server. Also we use Dovecot SASL for SMTP authentication. Because we are building some monitoring application, we will need to record all failed and successful login attempts. We need to record remote IP, entered password in plain text, and if possible whether auth request is for SMTP or IMAP session. I checked http://wiki.dovecot.org/PostLoginScripting and noticed that post-login scripts are executed only after result_success, but not after result_failure (password mismatch). Also I read http://wiki.dovecot.org/PasswordDatabase where I saw that since version 2.2.10 it is possible to control what happens after passdb check, but allowed result values don't include executing custom script. Does anyone know a way to call external binary / script, or at least save a record in the database after login attempt without reading the log files? P.S. there is also a special case. When someone logs in from webmail, remote IP is set to webmail's server. In this case, we will log the attempt from the webmail itself, because it has the correct remote IP. Robin Wood

2 1

Ox Dovecot - dovecot.fi - dovecot-ee repo access
by Raymond Sellars 23 Jan '17

23 Jan '17

Hi My question is a little off topic but hoping the community has some insight or advise. Has anyone had success in contacting OX dovecot. We previous purchased Dovecot Pro and had access to the dovecot-ee (https://yum.dovecot.fi) repository. Credentials are no longer valid and all attempts (over the months) to reach Ox gain no response. Wondering if others are having Pro access issues or have managed to reach their info/sales team (info(a)dovecot.fi<mailto:info@dovecot.fi>?). Any recommendations on alternatives for commercial support? Primarily active security patching/advisement rather than polling the open source release stream. Thanks Raymond

3 2

Re: Segfault on LIST Command
by Thorsten Hater 23 Jan '17

23 Jan '17

Hi, I did added the default location and stripped down my config to a very basic level, dropping all plugins and database queries, see below. The segfault still appears in the same location. As I have build from source, I wonder whether you can reproduce the problem? Thorsten $ doveconf -n # 2.2.26.0 (23d1de6): /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (1dc4c73) # OS: Linux 3.18.16-intel-vm-64bit x86_64 Debian 8.6 auth_debug = yes auth_debug_passwords = yes auth_socket_path = /usr/local/var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /usr/local/var/run/dovecot/ default_internal_user = pop first_valid_uid = 48 import_environment = TZ DEBUG=1 last_valid_uid = 48 login_greeting = Dovecot ready. login_trusted_networks = **** mail_debug = yes mail_gid = pop mail_location = maildir:~/Maildir mail_plugin_dir = /usr/local/lib/dovecot/ mail_uid = pop managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes list = children location = maildir:~/Maildir prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = nopassword=yes driver = static } protocols = imap pop3 lmtp imap pop3 ssl = no userdb { args = home=**** uid=pop gid=pop quota_rule=*:bytes=1000M driver = static } verbose_proctitle = yes protocol lda { auth_socket_path = /usr/local/var/run/dovecot/auth-userdb } On Mon, Jan 23, 2017 at 10:01 AM, Thorsten Hater <thorsten.hater(a)gmail.com> wrote: > Hi, > > thanks for picking this up. The location is pulled from the database, but > is uniform > for all users, so I could set it to maildir:~/Maildir globally. Assuming > ~ is expanded > later on with userdb data. So, no, there is no special intention behind > this. > > Thorsten > > On Mon, Jan 23, 2017 at 9:37 AM, Aki Tuomi <aki.tuomi(a)dovecot.fi> wrote: > >> >> >> On 19.01.2017 15:56, Thorsten Hater wrote: >> > The Problem arises due to a NULL deref in mail_namespaces.c line 601. >> > Backtrace below >> > >> > x LIST "" "" >> > >> > Program received signal SIGSEGV, Segmentation fault. >> > mail_namespaces_get_root_sep (namespaces=0x0) at mail-namespace.c:601 >> > 601 while ((namespaces->flags & NAMESPACE_FLAG_LIST_PREFIX) == 0) >> > (gdb) bt >> > #0 mail_namespaces_get_root_sep (namespaces=0x0) at >> mail-namespace.c:601 >> > #1 0x000000000041164c in cmd_list_ref_root (ref=0x65b060 "", >> > client=0x65a590) at cmd-list.c:324 >> > #2 cmd_list_full (cmd=0x65aee0, lsub=<optimized out>) at cmd-list.c:461 >> > #3 0x0000000000419825 in command_exec (cmd=cmd@entry=0x65aee0) at >> > imap-commands.c:181 >> > #4 0x0000000000417de2 in client_command_input (cmd=cmd@entry=0x65aee0) >> at >> > imap-client.c:988 >> > #5 0x0000000000417e70 in client_command_input (cmd=0x65aee0) at >> > imap-client.c:1048 >> > #6 0x00000000004181e5 in client_handle_next_command >> > (remove_io_r=<synthetic pointer>, client=0x65a590) at imap-client.c:1090 >> > #7 client_handle_input (client=0x65a590) at imap-client.c:1102 >> > #8 0x0000000000418692 in client_input (client=0x65a590) at >> > imap-client.c:1149 >> > #9 0x00007ffff76297ac in io_loop_call_io (io=0x652aa0) at ioloop.c:589 >> > #10 0x00007ffff762ab4a in io_loop_handler_run_internal >> > (ioloop=ioloop@entry=0x63e7f0) >> > at ioloop-epoll.c:222 >> > #11 0x00007ffff7629835 in io_loop_handler_run (ioloop=ioloop@entry >> =0x63e7f0) >> > at ioloop.c:637 >> > #12 0x00007ffff76299d8 in io_loop_run (ioloop=0x63e7f0) at ioloop.c:613 >> > #13 0x00007ffff75b9823 in master_service_run (service=0x63e690, >> > callback=callback@entry=0x423d40 <client_connected>) at >> master-service.c:641 >> > #14 0x000000000040c567 in main (argc=3, argv=0x63e390) at main.c:460 >> > >> > On Thu, Jan 19, 2017 at 1:05 PM, Thorsten Hater < >> thorsten.hater(a)gmail.com> >> > wrote: >> > >> >> Dear all, >> >> >> >> I experience SegFaults in the imap binary on a LIST "" "" command, >> >> as sent by Claws mail. Using LIST "" "INBOX" or similar is fine. >> >> Here is an example telnet session >> >> >> >> $ telnet 127.0.0.1 143 >> >> Trying 127.0.0.1... >> >> Connected to 127.0.0.1. >> >> Escape character is '^]'. >> >> * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE >> IDLE >> >> AUTH=PLAIN] Dovecot ready. >> >> 01 LOGIN **** **** >> >> 01 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE >> >> IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS >> THREAD=ORDEREDSUBJECT >> >> MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS >> >> LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES >> WITHIN >> >> CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in >> >> 02 LIST "" "" >> >> Connection closed by foreign host. >> >> >> >> In the log file >> >> >> >> dovecot[8375]: imap(***): Fatal: master: service(imap): child 15803 >> killed >> >> with signal 11 (core dumps disabled) >> >> >> >> Please find the config below. >> >> >> >> Best regards, >> >> Thorsten >> >> >> >> $ doveconf -n >> >> # 2.2.26.0 (23d1de6): /etc/dovecot/dovecot.conf >> >> # Pigeonhole version 0.4.16 (1dc4c73) >> >> # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 >> >> auth_debug = yes >> >> auth_debug_passwords = yes >> >> auth_socket_path = /var/run/dovecot/auth-userdb >> >> auth_verbose = yes >> >> base_dir = /var/run/dovecot/ >> >> default_internal_user = pop >> >> first_valid_uid = 48 >> >> import_environment = TZ DEBUG=1 >> >> last_valid_uid = 48 >> >> login_trusted_networks = **** >> >> mail_debug = yes >> >> mail_gid = pop >> >> mail_plugins = " mail_log notify zlib quota" >> >> mail_uid = pop >> >> managesieve_notify_capability = mailto >> >> managesieve_sieve_capability = fileinto reject envelope >> encoded-character >> >> vacation subaddress comparator-i;ascii-numeric relational regex >> imap4flags >> >> copy include variables body enotify environment mailbox date index >> ihave >> >> duplicate mime foreverypart extracttext >> >> namespace inbox { >> >> inbox = yes >> >> list = children >> >> location = >> >> mailbox Drafts { >> >> auto = no >> >> special_use = \Drafts >> >> } >> >> mailbox Sent { >> >> auto = no >> >> special_use = \Sent >> >> } >> >> mailbox Trash { >> >> auto = no >> >> autoexpunge = 30 days >> >> special_use = \Trash >> >> } >> >> mailbox drafts { >> >> auto = no >> >> special_use = \Drafts >> >> } >> >> mailbox sent { >> >> auto = no >> >> special_use = \Sent >> >> } >> >> mailbox spamverdacht { >> >> auto = no >> >> autoexpunge = 30 days >> >> special_use = \Junk >> >> } >> >> mailbox trash { >> >> auto = no >> >> autoexpunge = 30 days >> >> special_use = \Trash >> >> } >> >> mailbox virenverdacht { >> >> auto = no >> >> autoexpunge = 30 days >> >> special_use = \Junk >> >> } >> >> prefix = INBOX. >> >> separator = . >> >> subscriptions = yes >> >> type = private >> >> } >> >> passdb { >> >> args = nopassword=y >> >> driver = static >> >> } >> >> plugin { >> >> last_login_dict = file:~/lastlogin >> >> mail_log_events = delete undelete expunge copy mailbox_delete >> >> mailbox_rename >> >> mail_log_fields = uid box msgid size >> >> quota = maildir:User quota >> >> quota_warning = storage=80%% 80 %u %{userdb:quota_bytes} >> >> quota_warning2 = storage=90%% 90 %u %{userdb:quota_bytes} >> >> quota_warning3 = storage=95%% 95 %u %{userdb:quota_bytes} >> >> sieve = ldap:/etc/dovecot/pigeonhole-ldap.conf >> >> sieve_dir = ~/sieve >> >> sieve_plugins = sieve_storage_ldap >> >> zlib_save = gz >> >> zlib_save_level = 6 >> >> } >> >> service imap { >> >> executable = imap postlogin >> >> } >> >> service pop3 { >> >> executable = pop3 postlogin >> >> } >> >> service postlogin { >> >> executable = script-login -d rawlog >> >> } >> >> service quota-warning { >> >> executable = script /bin/quota-warning.sh >> >> } >> >> ssl = no >> >> userdb { >> >> args = /etc/dovecot/userdb-ldap.conf >> >> driver = ldap >> >> result_failure = return-fail >> >> result_internalfail = return-fail >> >> result_success = continue-ok >> >> } >> >> userdb { >> >> default_fields = quota_bytes=42M >> >> driver = bdb_quota >> >> override_fields = quota_rule=*:bytes=%{userdb:quota_bytes} >> >> result_failure = return-fail >> >> result_internalfail = return-fail >> >> result_success = continue-ok >> >> } >> >> verbose_proctitle = yes >> >> protocol lda { >> >> auth_socket_path = /var/run/dovecot/auth-userdb >> >> mail_plugin_dir = /lib/dovecot/modules >> >> mail_plugins = " mail_log notify zlib quota sieve" >> >> } >> >> protocol imap { >> >> mail_plugins = " mail_log notify zlib quota imap_xauth last_login >> >> imap_quota" >> >> } >> >> protocol pop3 { >> >> mail_plugins = " mail_log notify zlib quota last_login" >> >> } >> >> >> >> Hi! >> >> We are looking into this crash. >> >> Are you intentionally setting inbox namespace location to empty? >> >> Aki >> > >

2 6

Doveadm backup doesn't interpret placeholders
by James Pharaoh 23 Jan '17

23 Jan '17

Hi, Dovecot version: 2.2.27 I'm using replication between two servers for high availability, but I also want to backup all users. I have a setting for the replication to work like this: plugin { mail_replica = tcp:other-dovecot } This is fine, but then I also want to do a backup to a different directory for all users, since I am having trouble with an apparent bug running dovecot on btrfs. For this reason, I am storing my emails on ext4, but want to sync them to the BTRFS filesystem I use for my server before doing a snapshot. My mails are stored in /var/mail/mailboxes, and the backup is in /var/mail/mailboxes-backup, the former being the ext4 mount and the latter being part of the btrfs filesystem which belongs to the container where all this runs. The problem is that dovecot seems to only have a single concept of the mail backup location, which is either a remote, as above, or a filesystem location. The logical answer seems to be to provide the backup location when performing the local copy for a snapshot, so I run: doveadm backup -A mdbox:/var/mail/mailboxes-backup/%d/accounts/%u However, this is not picking up the %d and %u placeholders and replacing them with the values, as it would if configured with the option above. The only workaround I can find is to run doveadm backup individually for every user, with the full expanded path for each one. Am I missing something, or would it make sense for doveadm backup to simply recognise and respect these placeholders when passed on the command line? I've tried all sorts of other things, including -o mail_replica, and putting stuff in the userdb, but I can't find any other way to do this. Thanks in advance, James

2 3

Mailbox alias plugin and INBOX. folders
by James White 23 Jan '17

23 Jan '17

Hi, I'm trying to workaround the lack of IMAP folder mapping settings on Windows 10 Mobile in regards to controlling the Sent, Trash etc folder my device uses. Currently, my device creates its own "Sent Items" folder "Deleted Items" folder along with others, this is pain when using clients like Thunderbird which places such them in the correct places, because it can be configured this way. While this is a Microsoft issue, its looking very unlikely they will add IMAP mapping control, as its been like this since Windows Phone 7. So I was looking for performing a workaround at the server level. I came across the Dovecot Mailbox alias plugin, which sounds like a decent workaround, to at least allow sent email messages from my Windows 10 mobile device to show up in the right place, I'm aware of the potential for double cache etc, but I have enough storage for it. http://wiki2.dovecot.org/Plugins/MailboxAlias What I'm having difficulty with, is expressing the mailbox_alias to an existing INBOX. folder. My email server uses the INBOX prefix setup, so my configured folders are: INBOX\INBOX.Sent INBOX\INBOX.Drafts INBOX\INBOX.Trash INBOX\INBOX.Spam Most examples seem to indicate doing at the root folder level i.e. Sent Items -> Sent not Sent Items -> INBOX\INBOX.Sent. Is it possible to use the alias plugin in this way? If so how would I express this using mailbox_alias_old, I tried INBOX/Sent, but this seems to send all items to a blackhole, as they can no longer be found in any folder. It does appear to work at the root folder level, but as my setup is using the older INBOX prefix, I don't want to do that. Any hints/tips would be appreciated! Thanks, James

2 2

Segfault on LIST Command
by Thorsten Hater 23 Jan '17

23 Jan '17

Dear all, I experience SegFaults in the imap binary on a LIST "" "" command, as sent by Claws mail. Using LIST "" "INBOX" or similar is fine. Here is an example telnet session $ telnet 127.0.0.1 143 Trying 127.0.0.1... Connected to 127.0.0.1. Escape character is '^]'. * OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE AUTH=PLAIN] Dovecot ready. 01 LOGIN **** **** 01 OK [CAPABILITY IMAP4rev1 LITERAL+ SASL-IR LOGIN-REFERRALS ID ENABLE IDLE SORT SORT=DISPLAY THREAD=REFERENCES THREAD=REFS THREAD=ORDEREDSUBJECT MULTIAPPEND URL-PARTIAL CATENATE UNSELECT CHILDREN NAMESPACE UIDPLUS LIST-EXTENDED I18NLEVEL=1 CONDSTORE QRESYNC ESEARCH ESORT SEARCHRES WITHIN CONTEXT=SEARCH LIST-STATUS BINARY MOVE SPECIAL-USE QUOTA] Logged in 02 LIST "" "" Connection closed by foreign host. In the log file dovecot[8375]: imap(***): Fatal: master: service(imap): child 15803 killed with signal 11 (core dumps disabled) Please find the config below. Best regards, Thorsten $ doveconf -n # 2.2.26.0 (23d1de6): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (1dc4c73) # OS: Linux 3.16.0-4-amd64 x86_64 Debian 8.6 auth_debug = yes auth_debug_passwords = yes auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes base_dir = /var/run/dovecot/ default_internal_user = pop first_valid_uid = 48 import_environment = TZ DEBUG=1 last_valid_uid = 48 login_trusted_networks = **** mail_debug = yes mail_gid = pop mail_plugins = " mail_log notify zlib quota" mail_uid = pop managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace inbox { inbox = yes list = children location = mailbox Drafts { auto = no special_use = \Drafts } mailbox Sent { auto = no special_use = \Sent } mailbox Trash { auto = no autoexpunge = 30 days special_use = \Trash } mailbox drafts { auto = no special_use = \Drafts } mailbox sent { auto = no special_use = \Sent } mailbox spamverdacht { auto = no autoexpunge = 30 days special_use = \Junk } mailbox trash { auto = no autoexpunge = 30 days special_use = \Trash } mailbox virenverdacht { auto = no autoexpunge = 30 days special_use = \Junk } prefix = INBOX. separator = . subscriptions = yes type = private } passdb { args = nopassword=y driver = static } plugin { last_login_dict = file:~/lastlogin mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_warning = storage=80%% 80 %u %{userdb:quota_bytes} quota_warning2 = storage=90%% 90 %u %{userdb:quota_bytes} quota_warning3 = storage=95%% 95 %u %{userdb:quota_bytes} sieve = ldap:/etc/dovecot/pigeonhole-ldap.conf sieve_dir = ~/sieve sieve_plugins = sieve_storage_ldap zlib_save = gz zlib_save_level = 6 } service imap { executable = imap postlogin } service pop3 { executable = pop3 postlogin } service postlogin { executable = script-login -d rawlog } service quota-warning { executable = script /bin/quota-warning.sh } ssl = no userdb { args = /etc/dovecot/userdb-ldap.conf driver = ldap result_failure = return-fail result_internalfail = return-fail result_success = continue-ok } userdb { default_fields = quota_bytes=42M driver = bdb_quota override_fields = quota_rule=*:bytes=%{userdb:quota_bytes} result_failure = return-fail result_internalfail = return-fail result_success = continue-ok } verbose_proctitle = yes protocol lda { auth_socket_path = /var/run/dovecot/auth-userdb mail_plugin_dir = /lib/dovecot/modules mail_plugins = " mail_log notify zlib quota sieve" } protocol imap { mail_plugins = " mail_log notify zlib quota imap_xauth last_login imap_quota" } protocol pop3 { mail_plugins = " mail_log notify zlib quota last_login" }

2 2

mail-trends+Dovecot
by Odhiambo Washington 22 Jan '17

22 Jan '17

Hi, Out of curiosity, has anyone managed to use mail-trends[0] to analyse their e-mails. I thought about trying it out, but got stuck midstream. The mail-trends scripts work very well with gmail, but because they say it is supposed to work with _any_ IMAP server, I thought I could get it running with Dovecot too. I know this is NOT a mail-trends support group though, but I believe there is something minor that I am missing, which Dovecot admins could figure out. My idea was to see what comes out of my e-mails stored either in Maildir or mdbox. In my test env, I have mail_location = maildir:/var/spool/virtual/%d/%n/Maildir:INDEX=MEMORY I am testing with a virtual account, whose e-mails are in Maildir/ - some 30,000 mails. The gotcha is that I cannot figure out exactly what to specify in mail.py (for mail-trends [0]) for this situation. There is another modified version of mail-trends[1] which is supposed to be a lot easier than the original one, but this too doesn't seem to work in my setup: 1. My doveconf output -> http://bit.ly/2k11iKb 2. Accounts are virtual, so I have a user - johndoe(a)dom.ain who MUST login as johndoe(a)dom.ain with a password. 3. His mailbox is in /var/spool/virtual/dom.ain/johndoe/Maildir/ So I try this - using the mail-trends code from [1] root@gw:/scripts/mail-trends # ./main.py --server=localhost --username=johndoe(a)dom.ain --password=1234 --use_ssl [2017-01-21 22:24:08,552] Initializing {'username': 'johndoe(a)dom.ain', 'use_ssl': '', 'password': '1234', 'server': 'localhost'} [2017-01-21 22:24:08,552] Connecting [2017-01-21 22:24:08,747] Logging in [2017-01-21 22:24:08,752] Getting mailboxes *[2017-01-21 22:24:08,753] Found INBOX.spam *<-- Where it is getting this from beats me! *[2017-01-21 22:24:08,753] Found INBOX* Traceback (most recent call last): File "./main.py", line 304, in <module> message_infos = GetMessageInfos(opts) File "./main.py", line 104, in GetMessageInfos for mailbox in m.GetMailboxes(): File "/scripts/mail-trends/mail.py", line 61, in GetMailboxes mailboxes.remove('[Gmail]') ValueError: list.remove(x): x not in list root@gw:/scripts/mail-trends # root@gw:/scripts/mail-trends # *ls -al /var/spool/virtual/dom.ain/johndoe/Maildir/* total 3256 drwx------ 5 mailnull mailnull 512 Jan 22 00:04 . drwx------ 3 mailnull mailnull 512 Jan 21 18:48 .. drwx------ 2 mailnull mailnull 1084416 Jan 22 00:04 cur -rw------- 1 mailnull mailnull 1074769 Jan 21 18:52 dovecot-uidlist -rw------- 1 mailnull mailnull 8 Jan 21 18:32 dovecot-uidvalidity -r--r--r-- 1 mailnull mailnull 0 Jan 21 18:32 dovecot-uidvalidity.58837ef3 drwx------ 2 mailnull mailnull 1084416 Jan 21 18:47 new -rw-r--r-- 1 mailnull mailnull 0 Jan 21 18:30 subscriptions drwx------ 2 mailnull mailnull 512 Jan 21 17:44 tmp If anyone is able to hack this with Dovecot, please share the details. [0] http://code.google.com/p/mail-trends/ [1] https://github.com/jpbarraca/mail-trends -- Best regards, Odhiambo WASHINGTON, Nairobi,KE +254 7 3200 0004/+254 7 2274 3223 "Oh, the cruft."

2 2

Relative home path not allowed - but how is this relative?
by Roger Klorese 22 Jan '17

22 Jan '17

I just set up my server with MySQL support for authentication/authorization compatible with Postfix Admin. Initially, I was getting the "Relative home directory paths not supported" message, and it's easy to see why - Postfix Admin stores the maildir as "$domain/$userpart@$domain". But the directories in 10-mail.conf are set as mail_location = maildir:/home/mailboxes/%d/%u mail_home = maildir:/home/mailboxes/%d/%u ...and the user_query has been revised to: user_query = SELECT concat('/home/mailboxes/', maildir) as full_maildir FROM mailbox WHERE username = '%u'; ...so how are any of these relative paths?

1 1

Re: Mailbox problem on centos O/S after some "fiddling"
by Joseph Tam 21 Jan '17

21 Jan '17

> I do not understand why dovecot creates this folder as we ONLY ever use pop3 > (currently with plain text auth but going to secure when this sorted) > for mail and also why when it writes the blank folder it "owns" it with the > user accessing it (also the /var/mail/new folder takes the same owner) I think maybe those mailboxes were created because you've mapped all user home directories to the same folder (/var/mail), and thus, the first user to log in created files that subsequent users could not access. > mail_location = mbox:/var/mail/:INBOX=/var/mail/%u See this for the POP3 scenario you want: http://wiki.dovecot.org/MailLocation/mbox (Section: Only /var/mail mboxes) Don't need these > service imap-login { > inet_listener imap { > port = 143 > } > inet_listener imaps { > port = 993 > ssl = yes > } > } Joseph Tam <jtam.home(a)gmail.com>

1 0

Deleting a mailbox recursively
by Malte Schmidt 20 Jan '17

20 Jan '17

This topic was already discussed on this mailinglist: https://www.dovecot.org/list/dovecot/2011-November/079491.html I am also looking for a nice and convenient way of recursively deleting a mailbox which uses the mdbox-Format. Since then - was there any progress?

3 4

Replication between three servers
by James Pharaoh 19 Jan '17

19 Jan '17

Hi, Dovecot version: 2.2.27 I'm a little unsure if I have set up my replication correctly, and also how to set up replication on three servers, if this is possible. My replication works, with exactly the same configuration on both servers, except that the mail_replica contains the other server name in each case, of course. This seems to work fine. As I understand it, both need to be set up as a client and a server, since I have mail arriving on both. Is this a correct and supported configuration? I believe then I should be able to get three-way replication working, by pointing the three servers at each other in a circle. I am hoping this will cause a mail arriving at server 1 to be sent to server 2, which will then send it to server 3. Server 3 will attempt to send it back to server 1 which will already have it, and so this will terminate the loop. This is obviously going to create a bit of extra load, but I don't want to try it without some advice, and I'm not sure if the replication will be triggered properly by an email which is itself arriving via replication. Reading the description of the components, I think it should, since the aggregator service shouldn't care how an email has arrived, since it is simply reading the log, right? If a server fails, or needs to be taken out manually, I'll obviously need to reconfigure the loop, essentially taking it back to a standard two-server configuration. Similarly, I should be able to add a fourth server, and so on, if I wanted. I'm planning to distribute incoming emails over all servers, so that these will arrive and be stored on a working server during an outage without any reconfiguration, and to use something like nginx as a proxy to send each user to a specific server, to distribute the client load. Thanks in advance, James

1 0

Remove empty milder folders
by ＠lbutlr 18 Jan '17

18 Jan '17

Not sure if anyone will find this useful, but this is how I deleted a bunch (several hundred) of empty mail folders from a user account: # doveadm mailbox status -u “user(a)example.com" messages "*" ALL | grep "=0" | awk -F= '{print $1}' | awk '{print "rm -rf ."$1}' > list then I looked over list just to be sure it wasn’t mucked up and in the users root maildir: # sh < list and done. I probably could do some research on how to combine those two awk pipes but this worked, and it was fast. -- Apple broke AppleScripting signatures in Mail.app, so no random signatures.

2 2

Dynamic autoexpunge for IMAP folders
by Thorsten Hater 18 Jan '17

18 Jan '17

Dear all, is there a way to allow users to set up something like autoexpunge on an individual IMAP folder basis? The idea I implemented does not seem to work, namely returning namespace/inbox/mailbox/Bla/autoexpunge=6h from a userdb, which results in Debug: Unknown userdb setting: plugin/namespace/inbox/mailbox/Bla/autoexpunge=6h >From older questions on the mailing list I gather that the mailbox Bla needs to be set up in the config before I can override its fields. If there is a way around this, I would gladly use this. Another solution that comes to mind would be to use a table-driven cron job, but I would prefer the cleanup to be triggered at delivering and/or moving mail. Best regards, Thorsten

3 8

Plugin "mail_crypt" does not work
by Evgeniy Korneechev 18 Jan '17

18 Jan '17

Hi, i have a problem. I sent test e-mail. It is in folder "Sent", but it was not delivered (folder "Inbox" is empty). /var/log/dovecot: ......... lda(mail(a)example.com): Error: User initialization failed: mail_crypt_plugin: mail_crypt_global_public_key: Couldn't parse public key: Unknown key format ......... Try RSA and EC: https://wiki2.dovecot.org/Plugins/MailCrypt#RSA_key https://wiki2.dovecot.org/Plugins/MailCrypt#EC_key Why "Unknown key format"? RSA pubkey: -----BEGIN PUBLIC KEY----- MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQC4hfgRDlMJtN9rcV2VGa8gOF1g xiXHwokRkKmKfr64ZbqAhXzLzP8fSLo8ZEtRzfS3f/EyLRvYL9LHxlxYuSnq2LTW lbvSj8jcg3ucpA431Pbnq/OVI8WsXhXhZdipGcBDyzWHZw5Dp3I/am+FB96VlfPW maHO/oKGphXXhXSOXwIDAQAB -----END PUBLIC KEY----- EC pubkey: -----BEGIN PUBLIC KEY----- MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEw96p6hvv/BJi9njHyXq05ZrKA3RR BNFOslljkgv6YUb4de1pln6hTUN69CjMBEB5P3YI7KP8hooozwM8iN/wLQ== -----END PUBLIC KEY----- ------------------------------------------------ # 2.2.27 (c0f36b0): /etc/dovecot/dovecot.conf # Pigeonhole version 0.4.16 (fed8554) # OS: Linux 4.4.39-std-def-alt0.M80P.1 x86_64 ALT 8.1 Server auth_debug = yes auth_debug_passwords = yes auth_default_realm = example.com auth_mechanisms = plain login auth_socket_path = /var/run/dovecot/auth-userdb auth_verbose = yes auth_verbose_passwords = plain base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot disable_plaintext_auth = no first_valid_gid = 502 first_valid_uid = 502 last_valid_gid = 502 last_valid_uid = 502 log_path = /var/log/dovecot login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c mail_access_groups = vmail mail_debug = yes mail_gid = 502 mail_location = maildir:/%Lh/Maildir/:INDEX=/%Lh/Maildir/ mail_plugins = mail_crypt mail_privileged_group = vmail mail_uid = 502 managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext imapflags notify mbox_write_locks = fcntl namespace { list = children location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u prefix = shared/%%u/ separator = / subscriptions = yes type = shared } namespace inbox { inbox = yes location = prefix = separator = / type = private } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_crypt_curve = prime256v1 mail_crypt_global_private_key = </etc/dovecot/eckey/ecprivkey.pem mail_crypt_global_public_key = </etc/dovecot/eckey/ecpubkey.pem mail_crypt_save_version = 2 recipient_delimiter = + sieve = /etc/dovecot/sieves/default.sieve sieve_default = /etc/dovecot/sieves/default.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags sieve_max_redirects = 8 } protocols = imap pop3 lmtp sieve service auth-worker { user = root } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0666 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } } service imap { process_limit = 1024 vsz_limit = 256 M } service lmtp { unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } vsz_limit = 64 M } service pop3-login { inet_listener pop3 { port = 110 } } service pop3 { process_limit = 1024 } ssl_ca = /var/lib/ssl/cert.pem ssl_cert = </etc/dovecot/dovecot.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-ldap-user.conf.ext driver = ldap } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lmtp { mail_plugins = mail_crypt sieve } protocol lda { mail_plugins = mail_crypt sieve quota plugin { quota = dict:user::file:/var/vmail/glu_vrem/%n/.quotausage } } protocol imap { mail_plugins = mail_crypt autocreate quota imap_quota plugin { autocreate = INBOX autocreate2 = Sent autocreate3 = Trash autocreate4 = Drafts autocreate5 = Junk autosubscribe = INBOX autosubscribe2 = Sent autosubscribe3 = Trash autosubscribe4 = Drafts autosubscribe5 = Junk quota = dict:user::file:/var/vmail/glu_vrem/%n/.quotausage } } protocol sieve { mail_max_userip_connections = 10 managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } -- WBR, BaseALT/ALTLinux Team

2 6

Reappearing emails
by Ron Cleven 17 Jan '17

17 Jan '17

We have been running Dovecot 2.2.10 with a pair of CentOS 7 boxes with replications for the past year. We have been quite happy with the performance and reliability. Recently we received a report from an Android IMAP user that emails that he deleted without reading would often reappear in his INBOX. At first I thought it might be some quirky incompatibility with his email client, but I found that I could easily reproduce it in Roundcube by dragging an unread email to Trash. I have been monitoring this forum fairly religiously and don't recall seeing any similar reports. I plan to capture the IMAP wire interactions, but before doing that, I thought I would submit this to the IMAP gods to see if anyone had heard of this and/or if there is some newer version that might resolve the problem. This particular customer contacted his email vendor and asked them to analyze the problem. The actual log data is gigantic, so I just pasted in the vendor's narrative below in case it helps. Otherwise, as I said, I will capture a similar interaction at the wire level and follow up here. Vendor's narrative: You had 8 messages in the Inbox, and moved two to Trash. The server notified the app on the push connection (for Inbox) that there are now 6 messages. By the time the app got around to catch up with that (it did the Trash folder first), the server said "oh wait, there are 8 messages, not 6" (in the Inbox), and so the app synced them in, just as reported. app selecting the Inbox, server saying there are 8 messages there app moving UID 729 and 730 to Trash (push) server saying Trash now has 34 messages (push) server saying two messages gone from Inbox, 6 left (push) 6 messages left in Inbox becoming 8 (push) and here they are (the two new ones) Thanks, Ron

3 3

Mail rescue from Dovecot Server
by Joakim Magnussen - MediaLabs AS 17 Jan '17

17 Jan '17

Hi all, I have a mail folder from Dovecot server. Screenshot: https://i.stack.imgur.com/11NHs.jpg Is there any way to import to Outlook, or otherwise convert to readable format? I could setup Dovecot on my own server if there is any point.

2 1

Correct settings for ssl protocols" and "ssl ciphers"
by Jerry 17 Jan '17

17 Jan '17

I have the following two settings in my "10-ssl.conf" file # SSL protocols to use ssl_protocols = !SSLv2 # SSL ciphers to use ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL I have seen different configurations while Googling. I am wondering what the consensus is for the best settings for these two items. What do the developers recommend? Thanks! -- Jerry

4 3

fts-solr: Returning 400 on searches; unescaped braces
by Michael Welsh Duggan 17 Jan '17

17 Jan '17

Using Debian, dovecot-solr 1:2.2.26.0-4, and solr-tomcat 3.6.2+dfsg-9, I am getting 400 errors when doing searches. Here is an example search query from dovecot that failed (captured with wireshark): Frame 23: 338 bytes on wire (2704 bits), 338 bytes captured (2704 bits) on interface 0 Linux cooked capture Internet Protocol Version 6, Src: ::1, Dst: ::1 Transmission Control Protocol, Src Port: 56860, Dst Port: 8080, Seq: 1, Ack: 1, Len: 250 Hypertext Transfer Protocol GET /solr/select?fl=uid,score&rows=2664&sort=uid+asc&q={!lucene+q.op%3dAND}(hdr:test+OR+body:test)&fq=%2Bbox:6d5de009f991854df726000012cf7b9c+%2Buser:md5i HTTP/1.1\r\n Host: localhost:8080\r\n Date: Mon, 19 Dec 2016 00:25:56 GMT\r\n Connection: Keep-Alive\r\n \r\n [Full request URI: http://localhost:8080/solr/select?fl=uid,score&rows=2664&sort=uid+asc&q={!l…] [HTTP request 1/1] [Response in frame: 25] Here is the same query from firefox, which succeeds: Frame 66: 646 bytes on wire (5168 bits), 646 bytes captured (5168 bits) on interface 0 Linux cooked capture Internet Protocol Version 6, Src: ::1, Dst: ::1 Transmission Control Protocol, Src Port: 56862, Dst Port: 8080, Seq: 1, Ack: 1, Len: 558 Hypertext Transfer Protocol GET /solr/select?fl=uid,score&rows=2664&sort=uid+asc&q=%7B!lucene+q.op%3DAND%7D(hdr:test+OR+body:test)&fq=%2Bbox:6d5de009f991854df726000012cf7b9c+%2Buser:md5i HTTP/1.1\r\n Host: localhost:8080\r\n Connection: keep-alive\r\n Cache-Control: max-age=0\r\n Upgrade-Insecure-Requests: 1\r\n User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.75 Safari/537.36\r\n Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8\r\n DNT: 1\r\n Accept-Encoding: gzip, deflate, sdch, br\r\n Accept-Language: en-US,en;q=0.8\r\n \r\n [Full request URI: http://localhost:8080/solr/select?fl=uid,score&rows=2664&sort=uid+asc&q=%7B…] [HTTP request 1/1] [Response in frame: 86] The salient difference seems to be the encoding of the braces. Indeed in the tomcat 8 logs, I find the following which seems to corroborate my hypothesis: java.lang.IllegalArgumentException: Invalid character found in the request target. The valid characters are defined in RFC 7230 and RFC 3986 at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:467) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:667) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1437) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:745) Indeed the braces are listed in fts-backend-solr.c as part of solr_escape_chars, so don't know how the braces are making it through unencoded. -- Michael Welsh Duggan (md5i(a)md5i.com)

2 3

Dovecot source code audit
by Timo Sirainen 17 Jan '17

17 Jan '17

Mozilla sponsored source code audit for Dovecot. So thanks to them we have our first public code audit: https://wiki.mozilla.org/MOSS/Secure_Open_Source/Completed#dovecot Dates: October 2016 - January 2017 dovecot is a POP and IMAP mailserver; it is used in 68% of IMAP server deployments worldwide. The audit was performed by Cure53. The team found the following problems: • 3 Low The Cure53 team were extremely impressed with the quality of the dovecot code. They wrote: "Despite much effort and thoroughly all-encompassing approach, the Cure53 testers only managed to assert the excellent security-standing of Dovecot. More specifically, only three minor security issues have been found in the codebase, thus translating to an exceptionally good outcome for Dovecot, and a true testament to the fact that keeping security promises is at the core of the Dovecot development and operations."

9 9