- dovecot - dovecot.org

sieve script not applied
by Joan Aymà 15 Mar '15

15 Mar '15

Hi everyone, I installed all-new server postfix+amavis+dovecot and pigeonhole. Clients, thunderbird and roundcube creates sieve scripts ok and changed ones are seen by eatch other, so server at 4190 accepts the scripts and ensure they are created. But filters onto new mail isn't applied. Example: # rule:[guifinet] if header :contains :comparator "i;octet" "subject" "[guifi-" { fileinto "INBOX.guifinet"; } Isn't the rule correct? Didn't find any troubleshooting on this topic. Any ideas? Thanks.

1 0

Re: How to detect out-of-sync condition
by Joseph Tam 14 Mar '15

14 Mar '15

On Fri, 13 Mar 2015, Cliff Hayes writes: >> By closing off other avenues other than dovecot imap/pop/lda/etc., >> the indices will stay sync'd. > > I use dovecot's lda and dovecot's sieve filter. Then I'm not sure how mailboxes ever get out of sync. > So it looks like I need to compare the index/mailbox mtimes as you suggest. > > What am I looking for? > I see that the indexes are updated when I run the resync. > > I checked my mailbox (that was not resynced) and noticed that > dovecot.index last update was 16 days ago. Just to be clear, you're talking "doveadm index ...", yes? You might have to check dovecot.index.log* as well as they could contain the latest changes that have yet to propagate to the main index file. http://wiki2.dovecot.org/Design/Indexes/TransactionLog > So am I resyncing if the gap is over x days? > If so, is there a way to resync just those mailboxes with a doveadm > command or do I have to write a program look for that condition and run > doveadm when matched? It's possible that "doveadm index" checks modification times as an optimization measure. My recommended game plan: 1) Are your indices *really* out of date (checks logs as Steffen recommends)? 2) If so, how do they get out of date and can you avoid it? 3) If you can't avoid it, does it cause real problems? Most of the time, dovecot will seemlessly rebuild it and it's transparent to the user. 4) If you got this far, run a trial "doveadm index -A INBOX" (assuming you're just concerned about INBOXs) to see if it's really such a bad operation. As Knuth (hallowed be thy name!) said, "premature optimization is the root of all evil." Only after all these steps are found to be unsatisfactory would I consider writing your own scan and fix tool. Joseph Tam <jtam.home(a)gmail.com>

2 1

Re: How to detect out-of-sync condition
by Joseph Tam 14 Mar '15

14 Mar '15

Cliff Hayes writes: > I didn't see anything but the log is huge and I could have easily missed > it. > Is there a certain error or phrase I should look for? > If so please advise. In my setup using mbox, typical log entries would be like (Common) Warning: UIDVALIDITY changed ({int} -> {int}) in mbox file ... Error: Next message unexpectedly corrupted in mbox file ... (Rare) Error: Corrupted index cache file {cache-file}: Broken physical size for mail UID ... Error: read({file}) failed: Next message unexpectedly lost from mbox file ... Error: read({file}) failed: Cached message size larger than expected ... Joseph Tam <jtam.home(a)gmail.com>

1 0

JMAP support
by List 13 Mar '15

13 Mar '15

Just found http://jmap.io/ which is a JSON based RPC protocol for synchronizing messages/contacts/calendars. Any plans to support this protocol in Dovecot?

1 0

How to detect out-of-sync condition
by Cliff Hayes 13 Mar '15

13 Mar '15

Hello list, I recently had a user whose mailbox had gone out of sync. Webmail and client inbox totals had become vastly different and response times had become unacceptable. A doveadm force-resync fixed the problem nicely. This will no doubt happen again as we grow the service and I would like to have some type of indication at the server level so I can run a resync on a per-user basis as required. However, I don't know what to look for. I did not see any signs of the problem on the server - but perhaps I did not know what to look for. The only other alternative is for me to do a global resync in the cron.daily but my fear is that this won't scale well for thousands of users. Please advise. Thanks in advance.

3 6

Different realm for different listeners?
by Chris Adams 13 Mar '15

13 Mar '15

I see this has been asked a few times over the years (but not in several years), and the response was along the lines of "maybe someday", so I figured I'd see if someday was here yet... I have a need to have a different default realm for different listeners. Basically, I've a bunch of different domains, and a few of them insist their users not be required to authenticate with "user@domain" and just use "user", and I'm trying to make this work without separate daemons and/or VMs. I'll be using Dovecot 2.2 (most likely 2.2.10 as provided by RHEL/CentOS 7). Thanks. If there's not a way to do this directly in Dovecot, does anybody have any suggestion that doesn't require several different VMs? -- Chris Adams <cma(a)cmadams.net>

3 4

Active directory bind and quota stuff
by Fran 13 Mar '15

13 Mar '15

Hi list, first of all, please excuse me for any grammar mistake on my text, I'm still learning english. I'm making a Dovecot installation using my Active Directory like pass and user DB. I'm setting the per-user quota from an AD field (Initials) and everything seems to work fine, but I would like to know if you could help me to improve a couple of things. --------------------------------------------------------------------------- --------------------------------------------------------------------------- I'm using CentOS 7, this is my Dovecot build: --------------------------------------------------------------------------- # dovecot --build-options Build options: ioloop=epoll notify=inotify ipv6 openssl io_block_size=8192 Mail storages: shared mdbox sdbox maildir mbox cydir imapc pop3c raw fail SQL driver plugins: mysql postgresql sqlite Passdb: checkpassword ldap pam passwd passwd-file shadow sql Userdb: checkpassword ldap(plugin) nss passwd prefetch passwd-file sql # dovecot --version 2.2.10 And this is my /etc/dovecot/dovecot-ldap.conf.ext file: --------------------------------------------------------------------------- hosts = adserver1.dom, adserver2.dom base = DC=dom ldap_version = 3 auth_bind = yes auth_bind_userdn = dom\%u dn = cn=imapUser,cn=Users,dc=dom dnpass = ******** user_filter = (&(samaccountname=%u)(objectClass=person)(|(mail=%u@dom.com)(othermailbox=%u@dom.com))) pass_filter = (&(samaccountname=%u)(objectClass=person)(|(mail=%u@dom.com)(othermailbox=%u@dom.com))) pass_attrs = userPassword=password user_attrs = Initials=quota_rule=*:storage=%$MB --------------------------------------------------------------------------- --------------------------------------------------------------------------- My questions: 1) My AD doesn't allow anonymous searches and I was thinking in a way to avoid the need of have a specific user account with his password stored in a configuration file to do the bind. And I don't know if this is posible, but, does it exist a way to use the same user you are trying to validate to do bind with the AD server? 2) As you can see (user_attrs = Initials=quota_rule=*:storage=%$MB), to set the per-user quota value, I require a numeric value in "Initials" field of my AD. I've realized that if some wrong value is introduced in that field, Dovecot throws an error when you try to validate the user, and hence, imap logon fails. Does it exist any way to bypass that and to use the default quota value if something is wrong with the per-user quota set? Error log with wrong quota values (I wrote the value e34 in the Initials field of user1 in AD, instead of a valid numeric value): * quota refresh command: [root@dmail02 dovecot]# doveadm quota recalc -u user1 doveadm(user1): Error: user user1: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:storage=e34MB: Invalid rule limit value 'storage=e34MB': Unknown unit: e34MB * imap login: Mar 13 14:12:10 dmail02 dovecot: imap(user1): Error: user user1: Initialization failed: Failed to initialize quota: Invalid quota root quota: Invalid rule *:storage=e34MB: Invalid rule limit value 'storage=e34MB': Unknown unit: e34MB Mar 13 14:12:10 dmail02 dovecot: imap(user1): Error: Invalid user settings. Refer to server log for more information. Thank you very much in advance, Fran

1 0

Dovecot fails to compile --with-cdb (and attempted fix)
by Jeffrey 'jf' Lim 13 Mar '15

13 Mar '15

Dovecot version: 2.2.15 Tinycdb: 0.78 I'm attempting to compile dovecot --with-cdb, but keep getting into a problem with cdb: ======================== libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wstrict-aliasing=2 -Wl,--as-needed -o .libs/anvil main.o anvil-connection.o anvil-settings.o connect-limit.o penalty.o -Wl,--export-dynamic ../../src/lib-dovecot/.libs/libdovecot.so -ldl -lrt -Wl,-rpath -Wl,/usr/local/lib/dovecot ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_read' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_find' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_free' ../../src/lib-dovecot/.libs/libdovecot.so: undefined reference to `cdb_init' collect2: ld returned 1 exit status make[3]: *** [anvil] Error 1 make[3]: Leaving directory `/home/jf/dovecot-2.2.15/src/anvil' ========================== After some troubleshooting, I've finally managed to get dovecot to compile, albeit with a hack: --- src/lib-dovecot/Makefile.old 2015-03-09 17:36:15.690179802 +0000 +++ src/lib-dovecot/Makefile 2015-03-09 18:01:43.075719700 +0000 @@ -230,7 +230,7 @@ LIBDOVECOT_STORAGE_DEPS = $(top_builddir)/src/lib-storage/ libdovecot-storage.la $(top_builddir)/src/lib-imap-storage/ libimap-storage.la LIBICONV = LIBOBJS = -LIBS = -lrt +LIBS = -lrt -lcdb LIBTOOL = $(SHELL) $(top_builddir)/libtool LIBWRAP_LIBS = LINKED_STORAGE_LDADD = This isn't the best fix, admittedly (I wish I were more familiar with autotools), but I'll say that the configure process isn't right yet for cdb. Can anybody produce a better fix? Would love to learn more. thanks, -jf -- He who settles on the idea of the intelligent man as a static entity only shows himself to be a fool. Mensan / Full-Stack Technical Polymath / System Administrator 12 years over the entire web stack: Performance, Sysadmin, Ruby and Frontend

2 6

doveadm sync subscribed folders only
by Jernej Porenta 13 Mar '15

13 Mar '15

Heya, is there a simple way to use "doveadm sync" for subscribed folders only? Since the migration from UW-IMAP, our users can actually subscribe to non-mbox files and now, when we are pushing new servers out, we are migrating our mboxes to dbox format and we would like to migrate only subscribed folders. I know, there is a -m switch which can be used to transfer subscribed folders for each user, but it makes it quite unusable with escaping and all other problems with such volume, so just wondering if there is a simple way to migrate subscribed folders only. Thank you in advance, cheers, Jernej

1 0

Re: How to detect out-of-sync condition
by Cliff Hayes 13 Mar '15

13 Mar '15

Thanks. I use dovecot's lda and dovecot's sieve filter. So it looks like I need to compare the index/mailbox mtimes as you suggest. What am I looking for? I see that the indexes are updated when I run the resync. I checked my mailbox (that was not resynced) and noticed that dovecot.index last update was 16 days ago. So am I resyncing if the gap is over x days? If so, is there a way to resync just those mailboxes with a doveadm command or do I have to write a program look for that condition and run doveadm when matched? Cliff Hayes <chayes(a)afo.net> writes: > I recently had a user whose mailbox had gone out of sync. > Webmail and client inbox totals had become vastly different and response > times had become unacceptable. > A doveadm force-resync fixed the problem nicely. > > This will no doubt happen again as we grow the service and I would like to have some type of indication at the server level so I can run a resync on a per-user basis as required. Perhaps what you ought to look for is how are they getting out of sync in the first place. Assuming your webmail is a IMAP/POP client, I would look at ways that your user's INBOX can be modified out of Dovecot's sight. For example, does your LDA use dovecot's lda/lmtp, or do your users run mail clients that directly manipulate mailbox files, or do you use procmail/mail filters? By closing off other avenues other than dovecot imap/pop/lda/etc., the indices will stay sync'd. If you really have to do manual resyncing, comparing index/mailbox mtimes might be one way to do it. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: How to detect out-of-sync condition
by Joseph Tam 13 Mar '15

13 Mar '15

Cliff Hayes <chayes(a)afo.net> writes: > I recently had a user whose mailbox had gone out of sync. > Webmail and client inbox totals had become vastly different and response > times had become unacceptable. > A doveadm force-resync fixed the problem nicely. > > This will no doubt happen again as we grow the service and I would like > to have some type of indication at the server level so I can run a > resync on a per-user basis as required. Perhaps what you ought to look for is how are they getting out of sync in the first place. Assuming your webmail is a IMAP/POP client, I would look at ways that your user's INBOX can be modified out of Dovecot's sight. For example, does your LDA use dovecot's lda/lmtp, or do your users run mail clients that directly manipulate mailbox files, or do you use procmail/mail filters? By closing off other avenues other than dovecot imap/pop/lda/etc., the indices will stay sync'd. If you really have to do manual resyncing, comparing index/mailbox mtimes might be one way to do it. Joseph Tam <jtam.home(a)gmail.com>

1 0

doveconf -a Segmentation Fault
by Dan LaSota 13 Mar '15

13 Mar '15

Getting "Segmentation Fault" When I run doveconf -a Dan LaSota Instructional Designer, UAF eLearning (907) 451-4067 dan.lasota(a)alaska.edu http://elearning.uaf.edu

3 3

looks like a repeat of this problem
by Laeeth Isharc 12 Mar '15

12 Mar '15

http://permalink.gmane.org/gmane.mail.imap.dovecot/80062

1 0

location of dovecot.rawlog-directory
by Hardy Flor 12 Mar '15

12 Mar '15

How is it possible to have the dovecot.rawlog-directory outside the mail_location? A link to another directory is not because it is checked in rawlog.c S_ISDIR to directory.

3 4

rawlog with -i not worked
by Hardy Flor 12 Mar '15

12 Mar '15

Hello, I have the rawlog enabled according to the manual:http://wiki2.dovecot.org/Debugging/Rawlog Without -i everything is wonderful, a coredump is with the -i parameter created **************************************** service imap-postlogin { executable = script-login -d rawlog -i user = vmail } **************************************** GNU gdb (GDB) 7.4.1-debian Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/lib/dovecot/rawlog...(no debugging symbols found)...done. [New LWP 29312] warning: Can't read pathname for load map: Eingabe-/Ausgabefehler. [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". Core was generated by `/usr/lib/dovecot/rawlog -i /usr/lib/dovecot/script-login'. Program terminated with signal 11, Segmentation fault. #0 0x00007f66bd2fe6c7 in master_instance_list_find_by_name () from /usr/lib/dovecot/libdovecot.so.0 (gdb) bt full #0 0x00007f66bd2fe6c7 in master_instance_list_find_by_name () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #1 0x00007f66bd300ea5 in master_service_parse_option () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #2 0x00007f66bd300fd4 in master_getopt () from /usr/lib/dovecot/libdovecot.so.0 No symbol table info available. #3 0x0000000000401e15 in main () No symbol table info available. (gdb)

3 4

v2.2.16 release candidate released
by Timo Sirainen 12 Mar '15

12 Mar '15

http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz http://dovecot.org/releases/2.2/rc/dovecot-2.2.16.rc1.tar.gz.sig Looks like it's been a long time since v2.2.15. There have been a ton of changes since it was released though, so here's a release candidate first to find out if somebody can find any bugs before the final v2.2.16. Unfortunately I haven't had time/energy to read Dovecot mailing list for a while now. I'm hoping this will change, but I don't really expect it to happen anytime soon. On the positive side for Dovecot, it's now becoming used in more and more multi-million user installations, which brings all kinds of nice new improvements. The largest changes since v2.2.15: * dbox: Resyncing (e.g. doveadm force-resync) no longer deletes dovecot.index.cache file. The cache file was rarely the problem so this just caused unnecessary slowness. * Mailbox name limits changed during mailbox creation: Each part of a hierarchical name (e.g. "x" or "y" in "x/y") can now be up to 255 chars long (instead of 200). This also reduces the max number of hierarchical levels to 16 (instead of 20) to keep the maximum name length 4096 (a common PATH_MAX limit). The 255 char limit is hopefully large enough for migrations from all existing systems. It's also the limit on many filesystems. + director: Added director_consistent_hashing setting to enable consistent hashing (instead of the mostly-random MD5 hashing). This causes fewer user moves between backends when backend counts are changed, which may improve performance (mainly due to caching). + director: Added support for "tags", which allows one director ring to serve multiple backend clusters with different sets of users. + LMTP server: Added lmtp_user_concurrency_limit setting to limit how many LMTP deliveries can be done concurrently for a single user. + LMTP server: Added support for STARTTLS command. + If logging data is generated faster than it can be written, log a warning about it and show information about it in log process's process title in ps output. Also don't allow a single service to flood too long at the cost of delaying other services' logging. + stats: Added support for getting global statistics. + stats: Use the same session IDs as the rest of Dovecot. + stats: Plugins can now create their own statistics fields + doveadm server: Non-mail related commands can now also be used via doveadm server (TCP socket). + doveadm proxying: passdb lookup can now override doveadm_port and change the username. + doveadm: Search query supports now "oldestonly" parameter to stop immediately on the first non-match. This can be used to optimize: doveadm expunge mailbox Trash savedbefore 30d oldestonly + doveadm: Added "save" command to directly save mails to specified mailbox (bypassing Sieve). + doveadm fetch: Added body.snippet field, which returns the first 100 chars of a message without whitespace or HTML tags. The result is stored into dovecot.index.cache, so it can be fetched efficiently. + dsync: Added -t <timestamp> parameter to sync only mails newer than the given received-timestamp. + dsync: Added -F [-]<flag> parameter to sync only mails with[out] the given flag/keyword. + dsync: Added -a <mailbox> parameter to specify the virtual mailbox containing user's all mails. If this mailbox is already found to contain the wanted mail (by its GUID), the message is copied from there instead of being re-saved. (This isn't efficient enough yet for incremental replication.) + dsync: -m parameter can now specify \Special-use names for mailboxes. + imapc: Added imapc_features=gmail-migration to help migrations from GMail. See http://wiki2.dovecot.org/Migration/Gmail + imapc: Added imapc_features=search to support IMAP SEARCH command. (Currently requires ESEARCH support from remote server.) + expire plugin: Added expire_cache=yes setting to cache most of the database lookups in dovecot index files. + quota: If overquota-flag in userdb doesn't match the current quota usage, execute a configured script. + redis dict: Added support for expiring keys (:expire_secs=n) and specifying the database number (:db=n) - auth: Don't crash if master user login is attempted without any configured master=yes passdbs - Parsing UTF-8 text for mails could have caused broken results sometimes if buffering was split in the middle of a UTF-8 character. This affected at least searching messages. - String sanitization for some logged output wasn't done properly: UTF-8 text could have been truncated wrongly or the truncation may not have happened at all. - fts-lucene: Lookups from virtual mailbox consisting of over 32 physical mailboxes could have caused crashes.

4 7

Released Pigeonhole v0.4.7.rc1 for Dovecot v2.2.16.rc1
by Stephan Bosch 12 Mar '15

12 Mar '15

Hello Dovecot users, Last time I had a few stupid problems in the releases, so I'll follow Timo's example and I release an RC first. The highlights include the implementation of the index and metadata extensions. Quite a few bugs are fixed as well. Changelog v0.4.7: * editheader extension: Made protection against addition and deletion of headers configurable separately. Also, the `Received' and `Auto-Submitted' headers are no longer protected against addition by default. * Turned message envelope address parse errors into warnings. * The interpreter now accepts non-standard domain names, e.g. containing '_'. + Implemented the Sieve index extension (RFC 5260). + Implemented support for the mboxmetadata and servermetadata extensions (RFC 5490). + Implemented new sieve commands for the doveadm command line utility. These commands are currently limited to ManageSieve operations, but the other current sieve tools will be migrated to doveadm in the near future as well. + Added more debug output to binary up-to-date checking. - The Sieve interpreter now flushes duplicate database during start phase of result execution rather than commit phase. This makes sure locks on the duplicate database are released as soon as possible, preventing contention. - Performed a few optimizations in the lexical scanner of the language. - Fixed bug in `:matches' match-type that made a pattern without wildcards match as if there were a '*' at the beginning. - file storage: Restructured storage initialization to address backwards compatibility issues. - Fixed crash in validation of the string parameter of the comparator tag. - extprograms extension: Made sure supplemental group privileges are also dropped. This was a problem reported by Debian lintian. - Fixed bug in handling of binary errors for action side-effects and message overrides. The release is available as follows: http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1… http://pigeonhole.dovecot.org/releases/2.2/dovecot-2.2-pigeonhole-0.4.7.rc1… Refer to http://pigeonhole.dovecot.org and the Dovecot v2.x wiki for more information. Have fun testing this new release and don't hesitate to notify me when there are any problems. Regards, -- Stephan Bosch stephan(a)rename-it.nl

5 12

Expunge (?)
by Muzaffer Tolga Ozses 12 Mar '15

12 Mar '15

Hi, I don't know if this is the correct term. Let me elaborate. From time to time, I want to expunge (?) the mail folder so that users' Trash'd mails don't occupy space. So, here are my questions: 1. Is this the correct term? 2. Is it a Good Idea™ to do it? Thanks, -- mto

2 2

Sieve security: Any way to protect credentials used in extprograms?
by E.B. 11 Mar '15

11 Mar '15

I need to connect to a database in a script called using Sieve extprograms plugin. When delivering mail, Sieve is running as the mail recipient user, which means any files, either the sieve script or the extprograms it invokes, are run under that user's permissions. What would be a way to hide the database credentials in a more restricted file? I can think of... * Store the credentials in the database itself, return them from a DICT lookup that is fed to the sieve script. The credentials would have to be duplicated for every user record in the database. * Somehow put the value in the Sieve environment, but I'm not sure how to get a value into the environment (just return it in a userdb lookup?) and then how to get at that value once I'm executing the sieve script (can I retrieve it with the "environment" extension?) Are there better ways? [Of course, if I could do more elaborate things like call a stored procedure from Dovecot DICT then I could avoid this situation]

2 1

Sieve reject with ORIG-TO vs TO
by E.B. 11 Mar '15

11 Mar '15

Hi, The bounce message generated by the reject extension has what looks like a hard coded message prefix that comes before the configurable reason text: "Your message to <username> was automatically rejected:" In some cases, the <username> is NOT the original-to address, which can cause confusion to the sender or expose private aliasing data that some people might want to hide. Is there a way to make the reject extension use the original-to address in that hard-coded message prefix? Is there a way to completely customize the FULL message? Is there a way to customize the headers for such messages?

2 1

FTS search with CLucene giving odd results/exposing config issue
by David Gessel 11 Mar '15

11 Mar '15

I recently got FTS search with Clucene working on FreeBSD 10.1 (I had some trouble with 9.3, postponed solving it, then restarted with 10.1 by manually installing libtextcat, a listed dependency on http://wiki2.dovecot.org/Plugins/FTS/Lucene, but not part of the FreeBSD makefile: maybe that helped, maybe it was irrelevant). However, I now get some anomalous results. I'm putting them together as they may be related. Or, maybe not. 1) Searching from Thunderbird returns 35x as many results for the same term as searching from the command line. I'm not sure if the indexes are created or enumerable. 2) There seems to be a glitch in the maildir naming structure that results in a double slash (//) which doesn't seem to impact IMAP serving or Sieve, but maybe is causing the problem above with Clucene. Any hints? If I run a search from Thunderbird and "run search on server" for a somewhat infrequently used term like "titanium," I get results in more or less every archive folder one by one over about 20 minutes of searching and get about 516 results. It seems to run a little faster without the "run search on server" checkbox selected, but Thunderbird on Windows is abandonware and the only useful datapoint is 516 results. Maybe it is time for Claws. Anyway, if I run: # doveadm search -u user(a)domain.com mailbox '*' body titanium doveadm(user(a)domain.com): Error: stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory doveadm(user(a)domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal error occurred. Refer to server log for more information. [2015-03-11 09:11:16] bcadc922ba26b953a25701002812f29a 812478 bcadc922ba26b953a25701002812f29a 9338037 bcadc922ba26b953a25701002812f29a 9342918 bcadc922ba26b953a25701002812f29a 9343205 bcadc922ba26b953a25701002812f29a 9346096 bcadc922ba26b953a25701002812f29a 9350977 bcadc922ba26b953a25701002812f29a 9351379 bcadc922ba26b953a25701002812f29a 9352318 bcadc922ba26b953a25701002812f29a 9352441 bcadc922ba26b953a25701002812f29a 9352501 3d5d982e3a89b853c12c01002812f29a 1901940 3d5d982e3a89b853c12c01002812f29a 1908073 3d5d982e3a89b853c12c01002812f29a 1941453 3d5d982e3a89b853c12c01002812f29a 2036962 3d5d982e3a89b853c12c01002812f29a 2038360 I get 15 results. # doveadm fetch -u user(a)domain.com "mailbox" mailbox-guid 3d5d982e3a89b853c12c01002812f29a uid 1908073 mailbox: INBOX All are from the inbox. This takes roughly 30 seconds (much faster, that's good, though still not super fast) and in the logs I see promising messages like: Mar 11 09:27:52 host dovecot: indexer-worker(user(a)domain.com): Indexed 3708 messages in Commerce but also slightly less promising messages like: Mar 11 09:27:52 host dovecot: indexer-worker(user(a)domain.com): Warning: Maildir /mail/domain.com/user//.Bulkmail: Expunged message reappeared, giving a new UID (old uid=9335155, file=1426089637.M350915P91305.host,S=23845,W=24282:2,) If I execute # doveadm index -A '*' I get doveadm(user(a)domain.com): Error: stat(/mail/domain.com/user//.dovecot.sieve/tmp) failed: Not a directory doveadm(user(a)domain.com): Error: Syncing mailbox dovecot.sieve failed: Internal error occurred. Refer to server log for more information. [2015-03-11 09:51:26] (there's nothing in the server log) But note in both previous results a double slash (//) in the directory structure. I don't think that's right. So: Does # doveadm search search subfolders? The internets say either '*' or INBOX should search the hierarchy and the search command does set off indexing operations on every subfolder as indicated by the logs, but the results are only from the top level folder.... perhaps that is related to... Why do some errors report a double slash (//) in directory names? I don't seem to be having any other symptoms (if, indeed, this is one) of incompatibilities. My sieve filters work just fine and are accessible from the managesieve plugin in Thunderbird. My config file is below: # 2.2.15: /usr/local/etc/dovecot/dovecot.conf # Pigeonhole version 0.4.6 (3e924b1b6c5c+) # OS: FreeBSD 10.1-RELEASE amd64 auth_default_realm = domain.com auth_mechanisms = plain login auth_realms = domain.com domain2.org disable_plaintext_auth = no first_valid_gid = 5000 first_valid_uid = 5000 hostname = domain.com imap_idle_notify_interval = 29 mins last_valid_gid = 5000 last_valid_uid = 5000 lda_mailbox_autocreate = yes listen = * mail_gid = 5000 mail_location = maildir:~:CONTROL=/var/no-quota/%u mail_plugins = quota mail_log notify fts fts_lucene mail_temp_dir = /var/tmp mail_uid = 5000 maildir_broken_filename_sizes = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave duplicate spamtest spamtestplus editheader mbox_lazy_writes = no namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { expire = Trash fts = lucene fts_autoindex = yes fts_lucene = whitespace_chars=@. mime_parts mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir:User quota quota_exceeded_message = Storage quota for this account has been exceeded, please try again later. quota_rule = *:storage=50G quota_rule2 = Trash:storage=+30%% quota_rule3 = Sent:storage=+30%% quota_warning = storage=90%% quota-warning 90 %u quota_warning2 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /usr/local/etc/dovecot/sieve/ sieve_dir = ~/sieve sieve_extensions = +spamtest +spamtestplus +relational +comparator-i;ascii-numeric +editheader sieve_global_dir = /usr/local/etc/dovecot/sieve } postmaster_address = user(a)domain.com protocols = imap lmtp sieve sieve sendmail_path = /usr/local/sbin/sendmail service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0666 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 vsz_limit = 128 M } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { user = vmail } user = vmail } ssl = required ssl_cert = </usr/local/etc/ca/host-cert.pem ssl_key = </usr/local/etc/ca/host-unencrypted-key.pem ssl_prefer_server_ciphers = yes userdb { driver = prefetch } userdb { args = /usr/local/etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = quota mail_log notify fts fts_lucene sieve } protocol imap { mail_max_userip_connections = 20 mail_plugins = quota mail_log notify fts fts_lucene imap_quota imap_zlib } protocol lmtp { mail_plugins = quota mail_log notify fts fts_lucene sieve } (END)

1 0

libdriver_msql.so
by kaniggl 11 Mar '15

11 Mar '15

Hello, i installed dovecot 2.11.1 on ubuntu Linux dadd3041 3.16.0-30-generic #40-Ubuntu SMP Mon Jan 12 22:07:11 UTC 2015 ppc64le ppc64le ppc64le GNU/Linux To make it clear, architecture is PowerPC 64bit Then i installed dovecot via apt-get. But the file /usr/lib/dovecot/modules/auth/libdriver_mysql.so is missing. So i compiled dovecot myself on this system and got a file /usr/lib/dovecot/modules/auth/libdriver_mysql.so Then i restartetd the server and try to do a login via a webmailer (roundcube), but i get no connection to the database server. This is found in /var/log/mail.log Mar 11 15:19:04 dadd3041 dovecot: master: Dovecot v2.2.9 starting up (core dumps disabled) Mar 11 15:19:04 dadd3041 postfix/master[3367]: reload -- version 2.11.1, configuration /etc/postfix Mar 11 15:19:04 dadd3041 postfix/master[3367]: terminating on signal 15 Mar 11 15:19:04 dadd3041 postfix/master[3597]: daemon started -- version 2.11.1, configuration /etc/postfix Mar 11 15:19:25 dadd3041 dovecot: auth: Error: dlopen(/usr/lib/dovecot/modules/auth/libdriver_mysql.so) failed: /usr/lib/dovecot/modules/auth/libdriver_mysql.so: wrong ELF class: ELFCLASS32 Mar 11 15:19:25 dadd3041 dovecot: auth: Fatal: Unknown database driver 'mysql' Mar 11 15:19:25 dadd3041 dovecot: master: Error: service(auth): command startup failed, throttling for 2 secs Mar 11 15:19:25 dadd3041 dovecot: imap-login: Disconnected: Auth process broken (disconnected before auth was ready, waited 0 secs): user=<>, rip=::1, lip=::1, TLS handshaking, session=<rJpS9AMR9wAAAAAAAAAAAAAAAAAAAAAB> I know this means it is not a 64bit object but how could i achieve to get right one? best regards, kaniggl

3 4

dec2str ...
by Hardy Flor 11 Mar '15

11 Mar '15

a very stupid question: What reason is there for an output with printf until dec2str to convert the numeric value to a string and not to use the format identifier "%d" or "u%"?

2 1

full text index "per user"?
by Ralf Hildebrandt 11 Mar '15

11 Mar '15

Is there any way of disabling the creation of a full text index on a per user basis? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

1 1

Deduplicate Public Folder
by Bobber 11 Mar '15

11 Mar '15

How do you perform the doveadm deduplicate command on public folders? If I specify a user, nothing is found. Using -A also doesn't seem to work. -- *Bob Wooldridge* Blog: http://kc0dxf.net/blog/

2 1

Sieve extprograms socket vs. direct execution
by E.B. 11 Mar '15

11 Mar '15

Hi, I'm hoping to get some clarification of the differences between calling a script using the Sieve extprograms plugin execute method via direct execution or using the socket feature. Being naive, I see the socket option and think that way you tell Dovecot to spawn a daemon and I think that's going to be far superior in performance. But if that really was the difference, why would direct execution even be an option? (And doesn't the daemon still have to spawn a new shell every time if the target is a shell script? Does that defeat the purpose of having a daemon?) So I don't really know what the difference is, and under what circumstances you'd want to use one or the other. Can someone please help clarify?

1 0

Extending dict lookups (SQL)?
by E.B. 11 Mar '15

11 Mar '15

Hi, How difficult would it be to try to hack some extension to the Dovecot dict mechanism for someone unfamiliar with the code? I'm using SQL as a backend and am looking for, at a minimum, the ability to specify a WHERE clause in addition to the built-in one that feeds the current username. The field name to test the additional value against would be necessary as well of course. Naturally, full query customization would be great to have. Perhaps providing a mechanism for the caller to pass data for the extra WHERE clause is the hardest part, not sure... but it can't hurt to ask. Don't take this as lack of gratitude - I can work with what's there and greatly appreciate it.

1 0

Re: Outlook 2013/2010 nightmare #2
by Joseph Tam 11 Mar '15

11 Mar '15

"David.M.Clark" <david(a)davrom.com> writes: > So the bottom line for this particular site is: > > Set the "Root Folder" for IMAP in outlook to "mail". This is messy from > my beloved Linux command line perspective in that you end up with > ${HOME}/login_name/mail/mail. But it does work and stops the Outlook > crashes. If "root folder" is Outlook's parlance for IMAP prefix, you might find it helpful to configure a namespace alias. For examples, see http://wiki2.dovecot.org/Namespaces (Section Backwards Compatibility: UW-IMAP) It will map different npamespaces to the same folder so you don't have this mail/mail/ goofiness. > If you want to share a single e-mail account across multiple PCs running > Outlook 2013, you _cannot_ use the "Root Folder" of "mail" as I have > indicated above. The workaround is to create each subsequent PC with a > "mail2", "mail3" etc folder (without the quote marks of course). If you > set up two PCs with the same Root Folder, the new PC crashes out of > Outlook and eventually so does the original PC. The only way around this > is to delete the identity and PST files in Outlook and strictly set them > up again to different "mail" something folders. Almost reminds me of the > old MS "Share Violation" issue :-) > > So after the user is set to the mail2/mail3 folder and it appears under > the user's original "mail" folder, you then have to blow away the > mail2/mail3 folder and then do a symbolic link to the mail folder: > > ln -s mail mail2 Again, namespace aliases might help: you can configure as many as you like. It's a kludge though -- the behaviour you report is really bizarre. Joseph Tam <jtam.home(a)gmail.com>

1 0

Migrating from NetApp to ZFS
by Frido Otten 10 Mar '15

10 Mar '15

Hi All, We're currently in the process of migrating from a NetApp storage to a ZFS storage, both shared over NFS. We've ran into some problems which were solved by switching to dotlocks instead of flock. But now there's one problem left with the dotlock method. When connecting to Dovecot/IMAP, and moving messages from one folder to another, it is very slow and I'm suspecting this has all to do with the dotlock method. During this process of moving mail, the dotlock file is created and stays there for about 90 seconds. The mail client is unresponsive in this time period. This happens also when a new message is delivered to the mailbox. The dovecot example config talks about that dotlocking is using some tricks which may create more disk I/O than other locking methods. What are these tricks and can it be the cause of the slowness? We've tested it on different ZFS storages and the problem seems to only exist on ZFS storages which have their ZIL on SSD's Just creating and removing a file from the CLI is fast though. Regards, Frido

1 0

Re: Outlook 2013/2010 nightmare #2
by David.M.Clark 10 Mar '15

10 Mar '15

Yes Eric, Outlook also has a declaration that as of either version 2010 or 2013, they no longer download IMAP headers, they download the whole message - thank God for faster Internet connections these days - could you imagine that in the older dialup days? Still it is a waste of bandwidth and disk space to do this. I am so tired of how they claim to use the RFC and indicate they are 'compatible' with a certain protocol/procedure, but really just move the goal posts so you can only talk to M$ stuff at the back end using their enforcements. All part of locking people into the "perpetual upgrade" path I guess. On 03/10/2015 10:22 AM, Eric Broch wrote: > Not only does M$ Outlook not work well with others, it strips headers I > use for training spam filters. > > On 3/9/2015 5:48 PM, David.M.Clark wrote: >> Hi All and my sincere thanks to Jakob and Joseph for your responses. >> >> I got around the nightmare for this site but it is far from >> satisfactory and given both Thunderbird, Roundcube and the Android >> e-mail client work perfectly as expected, this following links >> comments enforce what I experienced over the weekend: >> >> http://comments.gmane.org/gmane.mail.imap.dovecot/79231 >> >> I have not struck this for other sites using Outlook 2013 (though I >> continually banter that Outlook version 'anything' is designed for MS >> Exchange and nothing else), but then again I have not had Outlook >> needing to share the same e-mail accounts across PCs. >> >> So the bottom line for this particular site is: >> >> Set the "Root Folder" for IMAP in outlook to "mail". This is messy >> from my beloved Linux command line perspective in that you end up with >> ${HOME}/login_name/mail/mail. But it does work and stops the Outlook >> crashes. >> >> If you set it to the Outlook recommended "INBOX" or "Inbox", you can't >> see or access subfolders. >> >> Now here comes the ugly part of Outlook 2013 using IMAP to Linux (at >> least for this site): >> >> If you want to share a single e-mail account across multiple PCs >> running Outlook 2013, you _cannot_ use the "Root Folder" of "mail" as >> I have indicated above. The workaround is to create each subsequent PC >> with a "mail2", "mail3" etc folder (without the quote marks of >> course). If you set up two PCs with the same Root Folder, the new PC >> crashes out of Outlook and eventually so does the original PC. The >> only way around this is to delete the identity and PST files in >> Outlook and strictly set them up again to different "mail" something >> folders. Almost reminds me of the old MS "Share Violation" issue :-) >> >> So after the user is set to the mail2/mail3 folder and it appears >> under the user's original "mail" folder, you then have to blow away >> the mail2/mail3 folder and then do a symbolic link to the mail folder: >> >> ln -s mail mail2 >> >> In this way, both users on the PCs will be seeing the same IMAP >> folders, being mail and mail2 on two different PCs. Naturally to add a >> third PC to this would be "mail3" etc. >> >> Now the other ugly part of this is you also need to update their >> .subscriptions file in ${HOME}/login_name/mail to show folders like: >> >> mail/Sent Items >> mail/Junk E-mails >> mail2/Sent Items >> mail2/Junk E-mails >> >> Given you would need to manually do this above, and is just not >> suitable, I have written two scripts to automate this process and it >> works cleanly so far. >> >> The first script does the automatic removal of the mail2/mail3 etc >> directory and creates the symbolic link, the second script is used to >> redo the .subscriptions file if the contents of the 'mail' directory >> change. >> >> You may need to mod these but I have a cron running as root to do this. >> >> Script 1 (mail_root_enforce): >> >> =----------------------------------------------------------------------------------= >> >> COMPLETE_HOME=/u/home >> cd ${COMPLETE_HOME} >> >> for USERHOME in * >> do >> ls -d ${USERHOME}/mail/mail? | while read CHECKME >> do >> JUST_CHECKME=`basename ${CHECKME}` >> if [ -d ${CHECKME} -a -L ${CHECKME} ] >> then >> #echo "${CHECKME} is ok" >> : >> else >> echo "${CHECKME} is _not_ ok" >> cd ${USERHOME}/mail >> rm -rf ${COMPLETE_HOME}/${CHECKME} >> ln -s mail ${JUST_CHECKME} >> fi >> done >> cd ${COMPLETE_HOME} >> done >> =----------------------------------------------------------------------------------= >> >> >> Script 2 (mail_multi_users): >> >> =----------------------------------------------------------------------------------= >> >> cd /u/home >> >> for THIS_USER in * >> do >> MORE_THAN_1=`ls -d ${THIS_USER}/mail/mail* | wc -l` >> if [ "${MORE_THAN_1}" -gt "1" ] >> then >> # echo "${THIS_USER} has more than one" >> ls ${THIS_USER}/mail/mail >${THIS_USER}/subcount >> if [ ! -f ${THIS_USER}/subcount_last ] >> then >> >${THIS_USER}/subcount_last >> fi >> ISDIFF=`diff ${THIS_USER}/subcount ${THIS_USER}/subcount_last | >> wc -l | awk '{print $1}'` >> if [ "${ISDIFF}" -gt "0" ] >> then >> # echo "Is Different" >> ls ${THIS_USER}/mail/mail >${THIS_USER}/subscription_tmp >> >${THIS_USER}/subscription_new >> for ROOT_MAIL_DIR in ${THIS_USER}/mail/mail* >> do >> MAIL_BASE_NAME=`basename ${ROOT_MAIL_DIR}` >> cat ${THIS_USER}/subscription_tmp | sed >> "s:^:$MAIL_BASE_NAME/:g" >>${THIS_USER}/subscription_new >> done >> cp ${THIS_USER}/subscription_new >> ${THIS_USER}/mail/.subscriptions >> chown ${THIS_USER}:popusers ${THIS_USER}/mail/.subscriptions >> cp ${THIS_USER}/subcount ${THIS_USER}/subcount_last >> fi >> fi >> done >> =----------------------------------------------------------------------------------= >> >> >> You will note the ${HOME} directories are all under /u/home so if you >> do sadly need to use these scripts, please change the /u/home to suite >> the ${HOME} location set on your server. I prefer to run this as root >> to enforce things and not have any funny permission errors, and also >> to save needing to run multiple instances for multiples users. >> >> One annoying caveat that the site will have to get used to: the mail2, >> mail3 folders do show up when using good e-mail clients like >> Thunderbird, SOGo, Roundcube or Android e-mail client. This is because >> they are seeing the contents of the .subscriptions file. I haven't >> looked into Dovecot doco for possible multiple .subscription files to >> make the folders bit look 'nicer'. I will look at something like >> 'namespace' to see if this might help. Please bear in mind this was in >> an emergency to get things at least 'working' so the users were happy >> when they hit the office on Monday. >> >> A documented story for your perusal and use if you are 'stuck' like I >> was. I will continue to try and see if there is a more elegant >> work-around but for now, this is all I can come up with. >> >> PS: I notice everyone suppresses their e-mail footer so doing the same >> here :-) >> -- As always, I remain at your service. Kindest Regards, David.M.Clark

3 2

Re: Outlook 2013/2010 nightmare #3
by David.M.Clark 10 Mar '15

10 Mar '15

Hi All, Sorry for the extra e-mail. Given I am helping roll out this identical setup again I am thinking I will test another approach which I am hoping will work but given having two or more PCs trying to access the 'mail' folder crashed at this site on the weekend, I went away from this. But in my thinking perhaps it 'might' work if same login ID is not used to access the exact same area. Example: 1. Setup user "accounts" and open the permissions of its folders. (${HOME} = /u/home/accounts) 2. Create "accounts2" but have its ${HOME} directory set to the same as "accounts". (${HOME} = /u/home/accounts) Make sure accounts2 can fully access everything - perhaps I will need to enforce a central GID (group ID) for each user to belong to (eg., accounts). 3. See if the same crash issue happens if using a different login name. This is using the reverse approach of not having separate Root Folder paths for the same Linux login ID. If this does work I can then reverse engineer the issue at the site over the weekend as well. Will keep you posted. -- As always, I remain at your service. Kindest Regards, David.M.Clark

1 0

Re: Outlook 2013/2010 nightmare #2
by David.M.Clark 10 Mar '15

10 Mar '15

Hi All and my sincere thanks to Jakob and Joseph for your responses. I got around the nightmare for this site but it is far from satisfactory and given both Thunderbird, Roundcube and the Android e-mail client work perfectly as expected, this following links comments enforce what I experienced over the weekend: http://comments.gmane.org/gmane.mail.imap.dovecot/79231 I have not struck this for other sites using Outlook 2013 (though I continually banter that Outlook version 'anything' is designed for MS Exchange and nothing else), but then again I have not had Outlook needing to share the same e-mail accounts across PCs. So the bottom line for this particular site is: Set the "Root Folder" for IMAP in outlook to "mail". This is messy from my beloved Linux command line perspective in that you end up with ${HOME}/login_name/mail/mail. But it does work and stops the Outlook crashes. If you set it to the Outlook recommended "INBOX" or "Inbox", you can't see or access subfolders. Now here comes the ugly part of Outlook 2013 using IMAP to Linux (at least for this site): If you want to share a single e-mail account across multiple PCs running Outlook 2013, you _cannot_ use the "Root Folder" of "mail" as I have indicated above. The workaround is to create each subsequent PC with a "mail2", "mail3" etc folder (without the quote marks of course). If you set up two PCs with the same Root Folder, the new PC crashes out of Outlook and eventually so does the original PC. The only way around this is to delete the identity and PST files in Outlook and strictly set them up again to different "mail" something folders. Almost reminds me of the old MS "Share Violation" issue :-) So after the user is set to the mail2/mail3 folder and it appears under the user's original "mail" folder, you then have to blow away the mail2/mail3 folder and then do a symbolic link to the mail folder: ln -s mail mail2 In this way, both users on the PCs will be seeing the same IMAP folders, being mail and mail2 on two different PCs. Naturally to add a third PC to this would be "mail3" etc. Now the other ugly part of this is you also need to update their .subscriptions file in ${HOME}/login_name/mail to show folders like: mail/Sent Items mail/Junk E-mails mail2/Sent Items mail2/Junk E-mails Given you would need to manually do this above, and is just not suitable, I have written two scripts to automate this process and it works cleanly so far. The first script does the automatic removal of the mail2/mail3 etc directory and creates the symbolic link, the second script is used to redo the .subscriptions file if the contents of the 'mail' directory change. You may need to mod these but I have a cron running as root to do this. Script 1 (mail_root_enforce): =----------------------------------------------------------------------------------= COMPLETE_HOME=/u/home cd ${COMPLETE_HOME} for USERHOME in * do ls -d ${USERHOME}/mail/mail? | while read CHECKME do JUST_CHECKME=`basename ${CHECKME}` if [ -d ${CHECKME} -a -L ${CHECKME} ] then #echo "${CHECKME} is ok" : else echo "${CHECKME} is _not_ ok" cd ${USERHOME}/mail rm -rf ${COMPLETE_HOME}/${CHECKME} ln -s mail ${JUST_CHECKME} fi done cd ${COMPLETE_HOME} done =----------------------------------------------------------------------------------= Script 2 (mail_multi_users): =----------------------------------------------------------------------------------= cd /u/home for THIS_USER in * do MORE_THAN_1=`ls -d ${THIS_USER}/mail/mail* | wc -l` if [ "${MORE_THAN_1}" -gt "1" ] then # echo "${THIS_USER} has more than one" ls ${THIS_USER}/mail/mail >${THIS_USER}/subcount if [ ! -f ${THIS_USER}/subcount_last ] then >${THIS_USER}/subcount_last fi ISDIFF=`diff ${THIS_USER}/subcount ${THIS_USER}/subcount_last | wc -l | awk '{print $1}'` if [ "${ISDIFF}" -gt "0" ] then # echo "Is Different" ls ${THIS_USER}/mail/mail >${THIS_USER}/subscription_tmp >${THIS_USER}/subscription_new for ROOT_MAIL_DIR in ${THIS_USER}/mail/mail* do MAIL_BASE_NAME=`basename ${ROOT_MAIL_DIR}` cat ${THIS_USER}/subscription_tmp | sed "s:^:$MAIL_BASE_NAME/:g" >>${THIS_USER}/subscription_new done cp ${THIS_USER}/subscription_new ${THIS_USER}/mail/.subscriptions chown ${THIS_USER}:popusers ${THIS_USER}/mail/.subscriptions cp ${THIS_USER}/subcount ${THIS_USER}/subcount_last fi fi done =----------------------------------------------------------------------------------= You will note the ${HOME} directories are all under /u/home so if you do sadly need to use these scripts, please change the /u/home to suite the ${HOME} location set on your server. I prefer to run this as root to enforce things and not have any funny permission errors, and also to save needing to run multiple instances for multiples users. One annoying caveat that the site will have to get used to: the mail2, mail3 folders do show up when using good e-mail clients like Thunderbird, SOGo, Roundcube or Android e-mail client. This is because they are seeing the contents of the .subscriptions file. I haven't looked into Dovecot doco for possible multiple .subscription files to make the folders bit look 'nicer'. I will look at something like 'namespace' to see if this might help. Please bear in mind this was in an emergency to get things at least 'working' so the users were happy when they hit the office on Monday. A documented story for your perusal and use if you are 'stuck' like I was. I will continue to try and see if there is a more elegant work-around but for now, this is all I can come up with. PS: I notice everyone suppresses their e-mail footer so doing the same here :-) -- As always, I remain at your service. Kindest Regards, David.M.Clark

1 0

Outlook 2013/2010 nightmare
by David.M.Clark 09 Mar '15

09 Mar '15

Hi All, This is my first post so forgive me if this hits the wrong list. I have been using dovecot for years happily with SendMail on mainly CentOS servers, and my customers are starting to more and more use Thunderbird, so my perfect model is: Linux --> Dovecot --> SendMail --> Thunderbird (or mobile phone e-mail app). I do have some customers using Outlook or Windows Live Mail, and these are for the most part working fine with IMAP - I don't do POP. My mod to the dovecot.conf file: disable_plaintext_auth=no My mod to conf.d/10-mail.conf: #mail_location = mail_location = mbox:~/mail:INBOX=/var/spool/mail/%u My mod to conf.d/10-master.conf: #default_internal_user = dovecot service imap-login { inet_listener imap { port = 143 } inet_listener imap_second { port = 10143 } inet_listener imaps { #port = 993 #ssl = yes } - I do the 10143 bit so I can pseudo hide port 143 for clients with devices that access both internally and externally - port forwarded on MikroTik routers for my customers. All goes well for many years with the above configs, and continues to be awesome as always. Under the user's ${HOME} directory they end up with a lovely 'mail' directory and all of the e-mail client folders live in harmony under there. This weekend I am helping a customer cut over his customer from legacy POP based e-mail uses with splatterings of Outlook 2010 and 2013 throughout the company, to using Linux with IMAP in the setup as described above as per my standard configs. The issue starts when you add an IMAP user to the Outlook client and upon opening it, initially, it tries to find a "Sent" Items folder under IMAP to send from. To this end I have traditionally gone into the dialogue box that opens, click on the IMAP account to receive a list of possible folders, but it now immediately crashes indicating Outlook has had an error, and it all goes to hell from there. Outlook then cannot open but keeps crashing with its "unknown error". Up to last week at another site with Windows Live Mail, this was not an issue. By modifying the e-mail account via Control Panel, I can delete or modify the account and try again. Here is what I found: If I don't specify a 'root' folder under the "Advanced" tab which also contains the port selections for 143 and 25, it fails. If I follow the MS recommended root folder example from Microsoft and use INBOX, or Inbox, it 'does' stop the error but you can't create subfolders - I am assuming as this is picking up the .imap/INBOX folder or something and makes sense that it can't create folders under INBOX/Inbox, particularly if this is a file rather than a directory that dovecot/Outlook are pointing to. What I have found is I can 'lie' to Outlook and put in a 'mail' folder. This results in a ${HOME}/login_name/mail/mail folder, but I don't care, as it happily puts files under mail/mail. Now here is something totally strange: If I try to put this same account on another PC so users can 'share' the same e-mail account, even with my 'mail' root folder work-around, it crashes again. All through this, Thuderbird works fine and so does RoundCube - SOGo will be up and working a bit later once I get the initial part up and going... but I digress but good to know it is only Outlook doing this. To share multiple accounts I am currently getting the subsequent PCs that need to share a universal account to select a root folder mail2, mail3 etc. I then delete these folders and symbolic link the mail2, mail3 etc to the mail directory - yucky fix, but kind of works with some auto scripting of .subscriptions to follow so all of the Outlook clients sync to the one 'real' root folder. I have never seen this before and have customers running all kinds of IMAP e-mail clients to dovecot on Linux. I saw an MS posted bug on something for IMAP and they recommend rolling back MS updates to fix it - but I am not sure the client can or will do that. This kind of random issue that only affects this site at present is why I only recommend people use e-mail clients like Thunderbird because it just 'works'. The users at this site would not switch and will cite they have been working in a POP situation on their other Linux box since Adam was a boy. Any help would be very much appreciated. I like to think I can do this with my eyes shut by now, but this one has got me stumped. -- As always, I remain at your service. Kindest Regards, David.M.Clark (Director - Senior Linux/UNIX Consultant) =----------------------------------------------------------------------= Davrom Consulting Pty Ltd E-mail : david(a)davrom.com PO Box 1644, Sunnybank Hills, 4109 Twitter: @DavidClark1961 ABN: 81 096 990 804 MSN: david(a)davrom.com Phone/Fax: 61-7-32720267 Skype: dmc1961 Mobile: 0418-763124 Google: dmc1961(a)gmail.com Podcast: http://www.davrom.com/ldup/ldup_rss.xml =---------------------- http://www.davrom.com -------------------------= Specialising in: Linux (Fedora/RedHat/CentOS), UNIX, SCO, MikroTik, Networking/Internet, E-mail/Web Technologies =----------------------------------------------------------------------= Please note: Any e-mail communication bearing this signature is for the exclusive purpose of the sender and is not for publication without the expressed permission of the sender or respective sender's organisation. =----------------------------------------------------------------------=

2 1

Expunge messages older than
by absolutely_free＠libero.it 09 Mar '15

09 Mar '15

Hi, I am using Dovecot 2.2.13 on cPanel server. Is there a way to delete messages older than some date for a number of users (in every subfolder)? Thank you

3 2

Re: Outlook 2013/2010 nightmare
by Joseph Tam 09 Mar '15

09 Mar '15

"David.M.Clark" <david(a)davrom.com> writes: > I do have some customers using Outlook or Windows Live Mail, and these > are for the most part working fine with IMAP - I don't do POP. > ... > The issue starts when you add an IMAP user to the Outlook client and > upon opening it, initially, it tries to find a "Sent" Items folder under > IMAP to send from. To this end I have traditionally gone into the > dialogue box that opens, click on the IMAP account to receive a list of > possible folders, but it now immediately crashes indicating Outlook has > had an error, and it all goes to hell from there. Outlook then cannot > open but keeps crashing with its "unknown error". > Up to last week at another site with Windows Live Mail, this was not an > issue. Just a shot in the dark, but have you enabled the workarounds? http://wiki2.dovecot.org/Clients Some clients like, like Outlook, have known problems which dovecot can dance around. in 20-imap.conf # Workarounds for various client bugs: # delay-newmail: # Send EXISTS/RECENT new mail notifications only when replying to NOOP # and CHECK commands. Some clients ignore them otherwise, for example OSX # Mail (<v2.1). Outlook Express breaks more badly though, without this it # may show user "Message no longer in server" errors. Note that OE6 still # breaks even with this workaround if synchronization is set to # "Headers Only". # tb-extra-mailbox-sep: # Thunderbird gets somehow confused with LAYOUT=fs (mbox and dbox) and # adds extra '/' suffixes to mailbox names. This option causes Dovecot to # ignore the extra '/' instead of treating it as invalid mailbox name. # tb-lsub-flags: # Show \Noselect flags for LSUB replies with LAYOUT=fs (e.g. mbox). # This makes Thunderbird realize they aren't selectable and show them # greyed out, instead of only later giving "not selectable" popup error. # # The list is space-separated. imap_client_workarounds = delay-newmail Joseph Tam <jtam.home(a)gmail.com>

1 0

dovecot auth-worker error happens when I enabled the dovecot to consider both the system account and the MySQL virtual mailbox databases.
by Emmanuel Chanel 08 Mar '15

08 Mar '15

Version: 2.2.9 & 2.2.15 ( http://ppa.launchpad.net/malte.swart/dovecot-2.2/ubuntu ) OS: Ubuntu 14.04 Server LTS I installed Ubuntu 14.04.2 LTS newly on my new server... And I try to configure the mail server considering both my system account and the virtual mailboxes, by seeing https://www.exratione.com/2014/05/a-mailserver-on-ubuntu-1404-postfix-dovec… But when I configured by uncommenting #!include auth-sql.conf.ext , it doesn't auth... Possibly, it's a trouble with the configurations of postfix and of dovecot. But I don't understand why... When I enable either auth-system.conf.ext or auth-sql.conf.ext , it works... My doveconf -n http://pastebin.com/MU1LL63d dovecot-sql.conf.ext http://pastebin.com/PJkTnPB3 tail /var/log/mail.log by debug mode / dovecot 2.2.15 http://pastebin.com/1TRQL49d Mar 6 07:45:12 gateway dovecot: auth: Debug: client in: AUTH#0114#011PLAIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333 Mar 6 07:45:12 gateway dovecot: auth: Debug: client passdb out: CONT#0114#011 Mar 6 07:45:12 gateway dovecot: auth: Debug: client in: CONT<hidden> Mar 6 07:45:12 gateway dovecot: auth-worker(22933): Debug: pam(emmanuel,192.168.0.2): lookup service=dovecot Mar 6 07:45:12 gateway dovecot: auth: Error: auth worker: Aborted PASSV request for emmanuel: Worker process died unexpectedly Mar 6 07:45:12 gateway dovecot: auth-worker(22936): Debug: sql(emmanuel,192.168.0.2): query: SELECT username as user, password, '/home/vmail//emmanuel' as userdb_home, 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' Mar 6 07:45:12 gateway dovecot: auth-worker(22933): Fatal: master: service(auth-worker): child 22933 killed with signal 11 (core dumped) Mar 6 07:45:12 gateway dovecot: auth-worker(22936): sql(emmanuel,192.168.0.2): unknown user Mar 6 07:45:12 gateway dovecot: auth-worker(23755): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 6 07:45:12 gateway dovecot: auth-worker(23755): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: FAIL#0114#011user=emmanuel#011temp Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: AUTH#0115#011LOGIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333 Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: CONT#0115#011VXNlcm5hbWU6 Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: CONT<hidden> Mar 6 07:45:14 gateway dovecot: auth: Debug: client passdb out: CONT#0115#011UGFzc3dvcmQ6 Mar 6 07:45:14 gateway dovecot: auth: Debug: client in: CONT<hidden> Mar 6 07:45:14 gateway dovecot: auth-worker(22936): Debug: pam(emmanuel,192.168.0.2): lookup service=dovecot Mar 6 07:45:14 gateway dovecot: auth: Error: auth worker: Aborted PASSV request for emmanuel: Worker process died unexpectedly Mar 6 07:45:14 gateway dovecot: auth-worker(23755): Debug: sql(emmanuel,192.168.0.2): query: SELECT username as user, password, '/home/vmail//emmanuel' as userdb_home, 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' Mar 6 07:45:14 gateway dovecot: auth-worker(22936): Fatal: master: service(auth-worker): child 22936 killed with signal 11 (core dumped) Mar 6 07:45:14 gateway dovecot: auth-worker(23755): sql(emmanuel,192.168.0.2): unknown user Mar 6 07:45:14 gateway dovecot: auth-worker(23764): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 6 07:45:14 gateway dovecot: auth-worker(23764): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Mar 6 07:45:16 gateway dovecot: auth: Debug: client passdb out: FAIL#0115#011user=emmanuel#011temp Mar 6 07:45:16 gateway dovecot: auth: Debug: client in: AUTH#0116#011PLAIN#011service=imap#011secured#011session=xS6uSpIQvQDAqAAC#011lip=192.168.0.1#011rip=192.168.0.2#011lport=143#011rport=38333#011resp=<hidden> Mar 6 07:45:16 gateway dovecot: auth-worker(23755): Debug: pam(emmanuel,192.168.0.2): lookup service=dovecot Mar 6 07:45:16 gateway dovecot: auth: Error: auth worker: Aborted PASSV request for emmanuel: Worker process died unexpectedly Mar 6 07:45:16 gateway dovecot: auth-worker(23764): Debug: sql(emmanuel,192.168.0.2): query: SELECT username as user, password, '/home/vmail//emmanuel' as userdb_home, 'maildir:/home/vmail//emmanuel' as userdb_mail, 65000 as userdb_uid, 8 as userdb_gid FROM mailbox WHERE username = 'emmanuel' AND active = '1' Mar 6 07:45:16 gateway dovecot: auth-worker(23755): Fatal: master: service(auth-worker): child 23755 killed with signal 11 (core dumped) Mar 6 07:45:16 gateway dovecot: auth-worker(23764): sql(emmanuel,192.168.0.2): unknown user Mar 6 07:45:16 gateway dovecot: auth-worker(23767): Debug: Loading modules from directory: /usr/lib/dovecot/modules/auth Mar 6 07:45:16 gateway dovecot: auth-worker(23767): Debug: Module loaded: /usr/lib/dovecot/modules/auth/libdriver_mysql.so Mar 6 07:45:18 gateway dovecot: auth: Debug: client passdb out: FAIL#0116#011user=emmanuel#011temp -- Emmanuel Chanel( emmanuelchanel(a)gmail.com )

1 3

Sieve can't find Extprograms or Extdata
by E.B. 08 Mar '15

08 Mar '15

Hi, On a new install-from-source with Dovecot 2.2.16rc1, Piegeonhole 0.4.6 and a grab of the newest Extdata code, I confirmed basic Sieve functionality is working (made a simple sieve script with a test on message subject resulting in a fileinto action). But I cannot get Sieve to see Extdata or Extprograms. sievec reports "Warning: sieve: ignored unknown extension 'vnd.dovecot.filter' while configuring available extensions" and the same for vnd.dovecot.extdata. And of course for the user script, "error: require command: unknown Sieve capability `vnd.dovecot.filter'" (same with extdata) I'm pretty sure this is a library path problem, but I've tried symlinking and restarting dovecot with as many iterations of library paths as I can think of but still same problem. The lib90_sieve_extdata_plugin.so and lib90_sieve_extprograms_plugin.so are in /usr/local/lib/dovecot/sieve and all the rest of the normal dovecot libraries are one level above in /usr/local/lib/dovecot. I tried creating symlinks for a "modules" directory and a "modules/sieve" directory, I tried symlinking these two librarie files themselves into the top level dovecot library directory, and a few other ideas, but no luck. Is there a way I can tell where Sieve expects its libraries to reside? To see where it is looking? I didn't provide any arguments to the configure command for pigeonhole as well as extdata and it seems like the libs got placed somewhere sensible, but I don't know what else to do.

2 3

[PATCH] Fix dovecot 1.2 build with LibreSSL
by Bernard Spil 07 Mar '15

07 Mar '15

Hi All, mail/dovecot build fails when linked against LibreSSL. This is due to LibreSSL no longer including comp.h from ssl.h/ssl3.h. See https://wiki.freebsd.org/LibreSSL as well. This patch fixes the build failure. Build log attached as well (not any more, too large! Get it via link just above). Please commit this fix to the 1.2 branch (2.1 branch is not affected) Kind regards, Bernard Spil.

1 0

Identify mail for attachment hash
by Daniel Miller 06 Mar '15

06 Mar '15

How can I find the message associated with a given attachment hash? For example, if I know that: /ff/23/ff23873c712474ad0f48cbd81485eaeaba3d8468-e1b4513a2eb8eb53b16400009db5accb is missing, I want to find the message associated with it so I can determine if it's critical and I need to recover it from another source, or if I can simply delete the old message. -- Daniel

1 0

Sieve editheader should support adding Received and Auto-Submitted headers
by Lee Maguire 06 Mar '15

06 Mar '15

In order to simplify auto-response suppression and other filtering, I want to synthesise an Auto-Submitted header on notification mails that should have them but do not. In a sieve file I add the following: if anyof ( header :contains ["From","Sender"] “info(a)example.com" ) { if header :contains "Subject" [" just joined "," just left "] { if not exists "Auto-Submitted" { addheader "Auto-Submitted" "auto-generated (via sieve)"; } } However this doesn’t work and results in the following error > main script: line 205: warning: addheader action: specified header field `Auto-Submitted' is protected; modification denied. (Workaround for this specific issue would probably be to use a List-* header instead.) Looking at http://hg.rename-it.nl/dovecot-2.2-pigeonhole/file/c8edece267cd/src/lib-sie… I can see that "Auto-Submitted” and “Received” are hardcoded to be blocked without reference to whether they are to be added or removed. However, the hard restriction in RFC 5293 on Auto-Submitted and Received only applies to the use of deleteheader and not addheader. https://tools.ietf.org/html/rfc5293#section-6 > As a matter of local policy, implementations MAY limit which header > fields may be deleted and which header fields may be added. However, > implementations MUST NOT permit attempts to delete "Received" and > "Auto-Submitted" header fields and MUST permit both addition and > deletion of the "Subject" header field. Using Dovecot version 2.1.7 but I assume it’s not version specific issue.

2 1

LMTP error: Too many concurrent deliveries for user (in reply to end of DATA command)
by Ralf Hildebrandt 06 Mar '15

06 Mar '15

I updated dovecot today and all over a sudden I'm getting: Mar 6 14:40:46 mail postfix/lmtp[3150]: 3kz95y3nX3zCtTS: to=<recipient(a)backup.invalid>, relay=127.0.0.1[private/dovecot-lmtp], delay=88, delays=87/0.94/0.01/0.01, dsn=4.3.0, status=deferred (host 127.0.0.1[private/dovecot-lmtp] said: 451 4.3.0 <recipient(a)backup.invalid> Too many concurrent deliveries for user (in reply to end of DATA command)) Why is that? Which setting must I tweak? -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

3 6

IP drop list
by Jim Pazarena 05 Mar '15

05 Mar '15

I wonder if there is an easy way to provide dovecot a flat text file of ipv4 #'s which should be ignored or dropped? I have accumulated 45,000+ IPs which routinely try dictionary and 12345678 password attempts. The file is too big to create firewall drops, and I don't want to compile with wrappers *if* dovecot has an easy ability to do this. If dovecot could parse a flat text file of IPs and drop connections it would sure put a dent in these attempts. Thanks.

15 28

Fwd: Re: IP drop list
by Jim Pazarena 05 Mar '15

05 Mar '15

On 2015-03-02 2:02 AM, Jochen Bern wrote: > On 03/01/2015 08:53 AM, Jim Pazarena wrote: >> I wonder if there is an easy way to provide dovecot a flat text file of >> ipv4 #'s which should be ignored or dropped? >> >> I have accumulated 45,000+ IPs which routinely try dictionary and >> 12345678 password attempts. The file is too big to create firewall >> drops [...] > > The inherent assumption here is that dovecot, using a "flat file", will > be able to process the block list more effectively than the firewall, > which is a tool written for the *purpose* but supposedly unable to even > *try* due to the list's size. That sounds ... counterintuitive. I am the original poster and just came back to this thread. When the first couple replies were "fail2ban" I lost interest. The reason I contemplated a flat text scan by dovecot is because, for the most part, my dovecot is low volume. So even if parsing a flat text file is less 'efficient' than a firewall insertion, it WOULD serve to defeat dictionary attacks rather readily. I already have a routine which scans my dovecot logs for goofy attacks such as dictionary or 12345 attempts. And since the attacks are pop/IMAP only, that is the only avenue which I wanted to defeat. This question garnered lots and lots of responses and I appreciate them all and read them all. And out of all the responses I think I will pursue the ipset routine. It seems easy enough and can act at the firewall level. The DNS RBL would be cool. I am also cognizant that 45,000 SHOULD have a TTL. However, these were IPs attempting to fetch email with obviously hacker type passwords. If, later, a given IP is re-assigned to a 'legitimate' person, they would still be able to send an email to me ' postmaster@ ' asking about an inability to fetch email. But parsing the flat text file would STILL be my preference. I'll look at the source and see if I can figure out where to inject such code. Like I said, my dovecot is low volume, so a fraction of a second at connection time is low impact. Considering that the flat text file may hang around in the memory cache it could even be less impact than low.

4 3

Receiving emails problem
by Gintare Ragaisiene 05 Mar '15

05 Mar '15

Hello, I have DirectAdmin system installed with CentOS OS. I created a user and a user email account. Then I tried to send email from my website (which is written on PHP5) to my mailbox. Everything seemed ok, but I did not received letter on the mailbox. The same is wehen I am trying to send through webmail SquirrelMail. I got a record on var/log/mailog: Mar 5 17:07:15 432282 dovecot[535]: imap-login: Login: user=<gintare>, method=PLAIN, rip=127.0.0.1, lip=127.0.0.1, mpid=7134, secured, session=<Rdpd7IsQhgB/AAAB> Mar 5 17:07:15 432282 dovecot[535]: imap(gintare): Disconnected: Logged out in=79 out=766 Please, help me. Versions: DirectAdmin - 1.4 CentOS - 6.6 Dovecot - 2.2.15 SquirrelMail (wbmail) - 1.4.23 Thanks, Gintare

2 1

Dovecot Full Text Search: HTTP 500 : Unknown fieldType 'text_general' specified on field text. [SERIOUS]
by Kevin Laurie 05 Mar '15

05 Mar '15

Hi Muzzafer, I get the error as specified below when i try to added it in as a field:- I dont think text_general is a valid field? HTTP ERROR 500 Problem accessing /solr/. Reason: {msg=SolrCore 'collection1' is not available due to init failure: Could not load conf for core collection1: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml,trace=org.apache.solr.common.SolrException: SolrCore 'collection1' is not available due to init failure: Could not load conf for core collection1: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml at org.apache.solr.core.CoreContainer.getCore(CoreContainer.java:745) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:307) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:207) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1419) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:455) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1075) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:384) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1009) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) at org.eclipse.jetty.server.Server.handle(Server.java:368) at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:942) at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:1004) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:640) at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:72) at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:264) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:608) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:543) at java.lang.Thread.run(Thread.java:745) Caused by: org.apache.solr.common.SolrException: Could not load conf for core collection1: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:66) at org.apache.solr.core.CoreContainer.create(CoreContainer.java:489) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:255) at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:249) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) ... 1 more Caused by: org.apache.solr.common.SolrException: Unknown fieldType 'text_general' specified on field text. Schema file is /opt/solr/solr/collection1/conf/schema.xml at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:595) at org.apache.solr.schema.IndexSchema.<init>(IndexSchema.java:166) at org.apache.solr.schema.IndexSchemaFactory.create(IndexSchemaFactory.java:55) at org.apache.solr.schema.IndexSchemaFactory.buildIndexSchema(IndexSchemaFactory.java:69) at org.apache.solr.core.ConfigSetService.createIndexSchema(ConfigSetService.java:90) at org.apache.solr.core.ConfigSetService.getConfig(ConfigSetService.java:62) ... 7 more Caused by: org.apache.solr.common.SolrException: Unknown fieldType 'text_general' specified on field text at org.apache.solr.schema.IndexSchema.loadFields(IndexSchema.java:638) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:489) ... 12 more ,code=500} On Thu, Mar 5, 2015 at 5:04 PM, Muzaffer Tolga Ozses <tolga(a)ozses.net> wrote: > Sure thing > > On 5 March 2015 at 11:52, Kevin Laurie <superinterstellar(a)gmail.com> wrote: >> >> No i dont have it. >> there is body field though. I think text is needed. Let me add it in and >> see. >> Thanks >> >> >> On Thu, Mar 5, 2015 at 4:42 PM, Muzaffer Tolga Ozses <tolga(a)ozses.net> >> wrote: >> > Leon gave you the answer. Check if you have it, define if you don't, and >> > restart the service. >> > >> > On 5 March 2015 at 11:24, Kevin Laurie <superinterstellar(a)gmail.com> >> > wrote: >> >> >> >> Hi Muzaffer, >> >> Could you help by checking this? Should I provide my solrconfig.xml >> >> too? >> >> Please advise urgently. I really need this to work man.Thanks! >> >> >> >> >> >> >> >> >> >>  >> >> <schema name="dovecot" version="1.5"> >> >> >> >> <field name="_version_" type="long" indexed="true" stored="true"/> >> >> <field name="_root_" type="string" indexed="true" stored="false"/> >> >> >> >> <types> >> >>  >> >> <fieldType name="string" class="solr.StrField" omitNorms="true"/> >> >> <fieldType name="long" class="solr.LongField" omitNorms="true"/> >> >> <fieldType name="slong" class="solr.SortableLongField" >> >> omitNorms="true"/> >> >> <fieldType name="float" class="solr.FloatField" omitNorms="true"/> >> >> <fieldType name="boolean" class="solr.BoolField" omitNorms="true"/> >> >> >> >> <fieldType name="text" class="solr.TextField" >> >> positionIncrementGap="100"> >> >> <analyzer type="index"> >> >> <tokenizer class="solr.WhitespaceTokenizerFactory"/> >> >> <filter class="solr.StopFilterFactory" ignoreCase="true" >> >> words="stopwords.txt"/> >> >> <filter class="solr.WordDelimiterFilterFactory" >> >> generateWordParts="1" generateNumberParts="1" catenateWords="1" >> >> catenateNumbers="1" catenateAll="0"/> >> >> <filter class="solr.LowerCaseFilterFactory"/> >> >>  >> >> <filter class="solr.SnowballPorterFilterFactory" >> >> protected="protwords.txt"/> >> >> <filter class="solr.NGramFilterFactory" minGramSize="3" >> >> maxGramSize="15"/> >> >> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> >> >> </analyzer> >> >> <analyzer type="query"> >> >> <tokenizer class="solr.WhitespaceTokenizerFactory"/> >> >> <filter class="solr.SynonymFilterFactory" >> >> synonyms="synonyms.txt" ignoreCase="true" expand="true"/> >> >> <filter class="solr.StopFilterFactory" ignoreCase="true" >> >> words="stopwords.txt"/> >> >> <filter class="solr.WordDelimiterFilterFactory" >> >> generateWordParts="1" generateNumberParts="1" catenateWords="0" >> >> catenateNumbers="0" catenateAll="0"/> >> >> <filter class="solr.LowerCaseFilterFactory"/> >> >>  >> >> <filter class="solr.SnowballPorterFilterFactory" >> >> protected="protwords.txt"/> >> >> <filter class="solr.RemoveDuplicatesTokenFilterFactory"/> >> >> </analyzer> >> >> </fieldType> >> >> </types> >> >> >> >> >> >> <fields> >> >> <field name="id" type="string" indexed="true" stored="true" >> >> required="true" /> >> >> <field name="uid" type="slong" indexed="true" stored="true" >> >> required="true" /> >> >> <field name="box" type="string" indexed="true" stored="true" >> >> required="true" /> >> >> <field name="user" type="string" indexed="true" stored="true" >> >> required="true" /> >> >> >> >> <field name="hdr" type="text" indexed="true" stored="false" /> >> >> <field name="body" type="text" indexed="true" stored="false" /> >> >> >> >> <field name="from" type="text" indexed="true" stored="false" /> >> >> <field name="to" type="text" indexed="true" stored="false" /> >> >> <field name="cc" type="text" indexed="true" stored="false" /> >> >> <field name="bcc" type="text" indexed="true" stored="false" /> >> >> <field name="subject" type="text" indexed="true" stored="false" /> >> >> </fields> >> >> >> >> <uniqueKey>id</uniqueKey> >> >> <defaultSearchField>body</defaultSearchField> >> >> <solrQueryParser defaultOperator="AND" /> >> >> </schema> >> >> >> >> >> >> On Thu, Mar 5, 2015 at 3:48 PM, Leon Kyneur <leon(a)dexterous.org> wrote: >> >> > In your schema.XML check you have defined: >> >> > >> >> > <field name="text" type="text_general" indexed="true" stored="false" >> >> > multiValued="true"/> >> >> > >> >> > On 05/03/2015 7:11 PM, "Kevin Laurie" <superinterstellar(a)gmail.com> >> >> > wrote: >> >> >> >> >> >> Hello, >> >> >> My dovecot constantly runs into this error. >> >> >> I want to fix this one last time, I am tired of troubleshooting so >> >> >> please someone give me a lasting and proper solution for this error. >> >> >> I >> >> >> think its a problem with the dovecot-solr module. >> >> >> >> >> >> Please tell me how do I find the root of this problem with Dovecot. >> >> >> There is a problem with the body search text field. It always >> >> >> fails(with no result), other searches work(ie. search date, subject >> >> >> etc, ) The field-text I believe is missing. Please help. Desperate >> >> >> here! >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> 2/25/2015, 11:32:30 PM ERROR SolrCore >> >> >> org.apache.solr.common. >> >> >> SolrException: undefined field text >> >> >> >> >> >> org.apache.solr.common.SolrException: undefined field text >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) >> >> >> at >> >> >> org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) >> >> >> at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) >> >> >> at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) >> >> >> at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) >> >> >> at >> >> >> >> >> >> org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) >> >> >> at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) >> >> >> at org.apache.solr.search.QParser.getQuery(QParser.java:141) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) >> >> >> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) >> >> >> at >> >> >> >> >> >> >> >> >> org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) >> >> >> at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) >> >> >> at java.util.concurrent.FutureTask.run(FutureTask.java:262) >> >> >> at >> >> >> >> >> >> >> >> >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> >> >> at >> >> >> >> >> >> >> >> >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> >> >> at java.lang.Thread.run(Thread.java:745) >> > >> > >> > >> > >> > -- >> > mto > > > > > -- > mto

2 2

Weird sub-folder error
by Christian Theune 05 Mar '15

05 Mar '15

Hi, I’m running dovecot 2.2.13 and I ran into a weird problem. A user migrated his mailboxes by letting users copy them from their old server to the new one through their MUA. All that went fine except for one very weird instance. The user is running Outlook 2010. The user has a sub-folder structure which dovecot places as dotted names beneath the Inbox. (E.g. Maildir/.my.sub.folders). After copying all data, that user came back to us and complained about missing mails in their folders. Apparently 3 of her folders disappeared. I found folders with similar (but different!) names in the dovecot storage but with no mails in them. After a while I noticed that something made dovecot create another level of nesting, like this: Maildir/.My.Folder/Subfolder In this case the Original folder structure was INBOX - My - Folder - Subfolder. When the user visited the subscription settings in her MUA she found the INBOX - My - Folder mailbox which was empty. The Subfolder was nowhere to be found. This appeared weird to me. I moved the Subfolder back directly under the Maildir directory and named it “.My.Folder.Subfolder”. doveadm now lists this folder correctly. I could not find whether this is a known bug - so I’m letting you know. Maybe you can help understand what happened. Sounds like a weird server-side bug to me - this shouldn’t happen, right? Cheers, Christian — Christian Theune · ct(a)flyingcircus.io · +49 345 219401 0 Flying Circus Internet Operations GmbH · http://flyingcircus.io Forsterstraße 29 · 06112 Halle (Saale) · Deutschland HR Stendal HRB 21169 · Geschäftsführer: Christian. Theune, Christian. Zagrodnick

2 4

Dovecot Full Text Search results in SolrException: undefined field text [SERIOUS]
by Kevin Laurie 05 Mar '15

05 Mar '15

Hello, My dovecot constantly runs into this error. I want to fix this one last time, I am tired of troubleshooting so please someone give me a lasting and proper solution for this error. I think its a problem with the dovecot-solr module. Please tell me how do I find the root of this problem with Dovecot. There is a problem with the body search text field. It always fails(with no result), other searches work(ie. search date, subject etc, ) The field-text I believe is missing. Please help. Desperate here! 2/25/2015, 11:32:30 PM ERROR SolrCore org.apache.solr.common. SolrException: undefined field text org.apache.solr.common.SolrException: undefined field text at org.apache.solr.schema.IndexSchema.getDynamicFieldType(IndexSchema.java:1269) at org.apache.solr.schema.IndexSchema$SolrQueryAnalyzer.getWrappedAnalyzer(IndexSchema.java:434) at org.apache.lucene.analysis.DelegatingAnalyzerWrapper$DelegatingReuseStrategy.getReusableComponents(DelegatingAnalyzerWrapper.java:74) at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:175) at org.apache.lucene.util.QueryBuilder.createFieldQuery(QueryBuilder.java:207) at org.apache.solr.parser.SolrQueryParserBase.newFieldQuery(SolrQueryParserBase.java:374) at org.apache.solr.parser.SolrQueryParserBase.getFieldQuery(SolrQueryParserBase.java:742) at org.apache.solr.parser.SolrQueryParserBase.handleBareTokenQuery(SolrQueryParserBase.java:541) at org.apache.solr.parser.QueryParser.Term(QueryParser.java:299) at org.apache.solr.parser.QueryParser.Clause(QueryParser.java:185) at org.apache.solr.parser.QueryParser.Query(QueryParser.java:107) at org.apache.solr.parser.QueryParser.TopLevelQuery(QueryParser.java:96) at org.apache.solr.parser.SolrQueryParserBase.parse(SolrQueryParserBase.java:151) at org.apache.solr.search.LuceneQParser.parse(LuceneQParser.java:50) at org.apache.solr.search.QParser.getQuery(QParser.java:141) at org.apache.solr.handler.component.QueryComponent.prepare(QueryComponent.java:148) at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:197) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1967) at org.apache.solr.core.QuerySenderListener.newSearcher(QuerySenderListener.java:64) at org.apache.solr.core.SolrCore$5.call(SolrCore.java:1739) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

3 3

Re: Re: IP drop list
by Joseph Tam 05 Mar '15

05 Mar '15

Nick Edwards writes: > I thought Timo once said dovecot had tarpitting, its useless if it is > there, and if it is, it needs user configurable timings, or maybe its > one of those things thats been in the gunna happen list > for a long time, like other stuff If I remember correctly, I think this was the "auth_failure_delay" feature. However, these delays are only inserted into the same session -- no IP tracking is done so a BFD attacking host can just keep opening up new connections. Dave McGuire writes: >>>> Please add this support to iptables instead of Dovecot. It's a >>>> waste of effort to code it into every application that listens on >>>> the network. >>> >>> <head explodes> >>> >>> Would you care to integrate it into IOS on my Cisco as well? >>> >>> There are things connected to the Internet that aren't PCs running >>> Linux, you know. It may be hard to accept, but that's the way it is. >>> >> I assume your dovecot runs on some kind of *nix > > Of course. I run it under Solaris. Oddly enough, if you run some versions of Solaris, it uses IPFilter as the native firewall, and it *does* have userland hooks so that you can make pass/block decisions based on userland executable. Not well documented though (see auth rules). You would need a firewall rule like auth in proto tcp from any to any port = 143 flags S/SA keep state then write a program that does ioctl(ipauth) calls to inspect and accept/reject packets; not for the faint of heart. However, you can use whatever weird and wonderful methods you want to determine IMAP/POP/SMTP network access policy, including DNSRBL or parsing a text file. This thread seems to be spinning into non-dovecot subjects, and I'm not helping, so I'll stop. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: IP drop list
by Joseph Tam 05 Mar '15

05 Mar '15

Dave McGuire writes: >> http://wiki2.dovecot.org/PasswordDatabase/ExtraFields/AllowNets >> >> then setup fail2ban to manage extrafields > > Now that's a very interesting idea, thank you! I will investigate this. If you don't expect yor firewall to handle 45K+ IPs, I'm not how you expect dovecot will handle a comma separated string with 45K+ entries any better. If you want to turn your global backlist into a per-user whitelist, that would be perfectly doable though. Joseph Tam <jtam.home(a)gmail.com>

14 31

Rebuilding SIS attachment links from log
by Daniel Miller 04 Mar '15

04 Mar '15

This seems simple enough...I'm just not script wizard. If someone can throw together a starting point I can test and tweak it from there. It seems to me: 1. Read /var/mail/mail.err or specified logfile 2. For each "failed: read(/var/mail/attachments/aa/bb/attachmentHash-userHash" line, a. Confirm /var/mail/attachments/aa/bb/hashes/attachmentHash exists i. If attachmentHash is missing display such for possible backup searching. b. create link attachmentHash-userHash to hashes/attachmentHash 3. Continue to end of file Can this be done via "pure" BASH? Need sed/awk as well? -- Daniel

3 3