- dovecot - dovecot.org

[Dovecot] dovecot lost mail! Cause?
by Lukas Haase 14 Nov '12

14 Nov '12

Hi, After using dovecot for several years now, today happend something which makes me really feel unconfortable: An email received was just not delivered properly, or, is lost! The mail (from an external server) was sent to two local mailboxes, user1 and user2. user1 received the message but for user2, it *magically* disappeared. MTA is exim4 which definitely processed the messages and handed over to dovecot deliver: 2012-11-12 07:28:21 1TXnVG-00053I-GD SA: [...] id=8644593.887351352701 685934.JavaMail 2012-11-12 07:28:21 1TXnVG-00053I-GD => user1 <user1(a)example.com> R=dovecot T=dovecot_pipe 2012-11-12 07:28:21 1TXnVG-00053I-GD => user2 <user2(a)example.com> R=dovecot T=dovecot_pipe 2012-11-12 07:28:21 1TXnVG-00053I-GD Completed Also, the log of dovecot tells that the mail should have been stored: Nov 12 07:28:21 mail dovecot: deliver(user1): sieve: msgid=<8644593.887351352701685934.JavaMail>: stored mail into mailbox 'INBOX' Nov 12 07:28:21 mail dovecot: deliver(user2): sieve: msgid=<8644593.887351352701685934.JavaMail.orbitz>: stored mail into mailbox 'INBOX' user1 received the mail but user2 not. Since user2 is myself, I *know* that I did not accidently delete any mail or something like that. It was just never received! Disk space is 3GB left, so enough. So I grepped the whole Maildir of user2 for the message ID. There is only one match in the dovecot.index.cache and within that, the most important data of the mail (Message ID, Date, Sender, Receiver, Subject) appears. But apart from that, not a single file! Is there hope to ever find out why what was going wrong here? It feels me veeeeeeeeery unconfortable because from now on I can never be sure any more that all my mails are really received :( :( However, as I said, my mail system processed maybe millions of messages the past 8 years. Although, I can not be sure if that was the case ... :( And help greatly appreciated! Luke PS: Dovecot version 1.2.15 (Debian 6.0.6) with Maildir backend on local harddrive. No NFS, nothing which can go wrong ... PPS: Original log files, just named replaced for privacy.

4 5

[Dovecot] Dovecot newbie (migrating from qmail)
by Ajai Khattri 13 Nov '12

13 Nov '12

Ive been using qmail+vpopmail+courier-imap for many years but its time to retire that server so I thought this might be an opportunity to see how I could use Postfix and Dovecot to provide the same services. Im running Dovecot 2.1.9 and Postfix 2.9.4. I have spent a few days reading through some of the docs on the wiki (there's doesn't seem to be any large overview of Dovecot concepts or books so bear with me). Im thinking something simple like passwd-file setup would suffice for me. I want to use completely virtual users. Id like to store mail under /home/vmail/$domain/$user. I think Ive figured out how to do that. But how do I create the maildir? I understand the need to have a mail directory but also a directory for things like sieve - how to specify that? Im thinking the structure would be something like: /home/vmail/$domain/$user/mail <-- mail stored here /home/vmail/$domain/$user/ <-- sieve and other sundry store here Or should the sieve stuff also be under its own folder alongside the maildir? Is it possible to have a separate passwd file per domain? It is possible to combine password and userdb files into one per domain? (I prefer to keep all files related to each domain in its own folder). Also would like to configure Postfix to use dovecot-sasl and I want to use dovecot-lmtp for deliveries. Any good docs / example on those? Later on, I think I want to integrate some kind of virus and spam filtering (amavisd and maybe dspam - anybody used those? would I configure Postfix to handle that or dovecot-lmtp?) Im sure Ill have more questions as I get more complicated but baby steps for now. -- Aj.

3 5

[Dovecot] Problem importing messages with lazy_expunged configured
by Angel L. Mateo 13 Nov '12

13 Nov '12

Hello, I have a problem importing messages since I have configured lazy_expunge. My lazy_expunge configuration (that is correctly working when a message is expunged) is: plugin { lazy_expunge = BORRADOS. } namespace { hidden = yes list = no location = maildir:~/Maildir/expunged prefix = BORRADOS. separator = . } but when I try to import messages I get: amateo_adm@myotis31:~$ sudo doveadm -vD import -u angel.luis maildir:/tmp/angel.luis '' mailbox INBOX doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib02_lazy_expunge_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib10_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/lib20_zlib_plugin.so doveadm(root): Debug: Loading modules from directory: /usr/lib/dovecot/modules/doveadm doveadm(root): Debug: Skipping module doveadm_acl_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_acl_plugin.so: undefined symbol: acl_user_module (this is usually intentional, so just ignore this message) doveadm(root): Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_quota_plugin.so doveadm(root): Debug: Module loaded: /usr/lib/dovecot/modules/doveadm/lib10_doveadm_zlib_plugin.so doveadm(root): Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib/dovecot/modules/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_backend_rescan (this is usually intentional, so just ignore this message) doveadm(root): Debug: Effective uid=0, gid=0, home=/home/amateo_adm doveadm(root): Debug: Quota root: name=User quota backend=dict args=:file:/home/amateo_adm/Maildir/dovecot.quota doveadm(root): Debug: Quota rule: root=User quota mailbox=* bytes=10737418240 messages=0 doveadm(root): Debug: Quota rule: root=User quota mailbox=Trash bytes=+1073741824 messages=0 doveadm(root): Debug: dict quota: user=doveadm, uri=file:/home/amateo_adm/Maildir/dovecot.quota, noenforcing=0 doveadm(root): Debug: maildir++: root=/tmp/angel.luis, index=, control=, inbox=/tmp/angel.luis, alt= doveadm(root): Fatal: lazy_expunge: Unknown namespace: 'BORRADOS.' in /tmp/<user> I don't have any "expunged" folder, it is just a maildir. In order to use it I have to deactivate this plugin in the running command with something like "sudo doveadm -vD -o mail_plugins='quota zlib' import -u <user> maildir:/tmp/<user> '' mailbox INBOX". Any idea? -- Angel L. Mateo Martínez Sección de Telemática Área de Tecnologías de la Información y las Comunicaciones Aplicadas (ATICA) http://www.um.es/atica Tfo: 868889150 Fax: 868888337

1 0

[Dovecot] older gcc don't know --as-needed
by Andreas Schulze 13 Nov '12

13 Nov '12

Hi, I have to build dovecot-2.1.10 for SuSE SLES9. /bin/sh ../../libtool --tag=CC --mode=link gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -no-undefined -Wl,--as-needed -o test-lib test_lib-test-lib.o test_lib-test-array.o test_lib-test-aqueue.o test_lib-test-base64.o test_lib-test-bsearch-insert-pos.o test_lib-test-buffer.o test_lib-test-crc32.o test_lib-test-hash-format.o test_lib-test-hex-binary.o test_lib-test-istream-base64-encoder.o test_lib-test-istream-concat.o test_lib-test-istream-crlf.o test_lib-test-istream-seekable.o test_lib-test-istream-tee.o test_lib-test-llist.o test_lib-test-mempool-alloconly.o test_lib-test-network.o test_lib-test-ostream-file.o test_lib-test-primes.o test_lib-test-priorityq.o test_lib-test-seq-range-array.o test_lib-test-strescape.o test_lib-test-strfuncs.o test_lib-test-str-find.o test_lib-test-str-sanitize.o test_lib-test-time-util.o test_lib-test-utc-mktime.o test_lib-test-var-expand.o ../lib-test/libtest.la liblib.la -lrt libtool: link: gcc -std=gnu99 -g -O2 -Wall -W -Wmissing-prototypes -Wmissing-declarations -Wpointer-arith -Wchar-subscripts -Wformat=2 -Wbad-function-cast -fno-builtin-strftime -Wl,--as-needed -o test-lib test_lib-test-lib.o test_lib-test-array.o test_lib-test-aqueue.o test_lib-test-base64.o test_lib-test-bsearch-insert-pos.o test_lib-test-buffer.o test_lib-test-crc32.o test_lib-test-hash-format.o test_lib-test-hex-binary.o test_lib-test-istream-base64-encoder.o test_lib-test-istream-concat.o test_lib-test-istream-crlf.o test_lib-test-istream-seekable.o test_lib-test-istream-tee.o test_lib-test-llist.o test_lib-test-mempool-alloconly.o test_lib-test-network.o test_lib-test-ostream-file.o test_lib-test-primes.o test_lib-test-priorityq.o test_lib-test-seq-range-array.o test_lib-test-strescape.o test_lib-test-strfuncs.o test_lib-test-str-find.o test_lib-test-str-sanitize.o test_lib-test-time-util.o test_lib-test-utc-mktime.o test_lib-test-var-expand.o ../lib-test/.libs/libtest.a ./.libs/liblib.a -lrt /usr/lib/gcc-lib/i586-suse-linux/3.3.3/../../../../i586-suse-linux/bin/ld: unrecognized option '--as-needed' /usr/lib/gcc-lib/i586-suse-linux/3.3.3/../../../../i586-suse-linux/bin/ld: use the --help option for usage information gcc version is 3.3.3 Is there a special option to disable ithe ld option or do I have to apply a special patch? Are there other suggestions? Thanks -- Andreas Schulze Internetdienste | P252 DATEV eG 90329 Nürnberg | Telefon +49 911 319-0 | Telefax +49 911 319-3196 E-Mail info @datev.de | Internet www.datev.de Sitz: 90429 Nürnberg, Paumgartnerstr. 6-14 | Registergericht Nürnberg, GenReg Nr.70 Vorstand Prof. Dieter Kempf (Vorsitzender) Dipl.-Kfm. Wolfgang Stegmann (stellvertretender Vorsitzender) Dipl.-Kfm. Michael Leistenschneider Dipl.-Kfm. Dr. Robert Mayr Jörg Rabe v. Pappenheim Dipl.-Vw. Eckhard Schwarzer Vorsitzender des Aufsichtsrates: Reinhard Verholen

1 0

[Dovecot] Commercial features in Dovecot future: Object storage, archive
by Timo Sirainen 13 Nov '12

13 Nov '12

Hi all, Dovecot Oy’s web pages at www.dovecot.fi have been updated. The products page lists two features that will be available for commercial licensing, extending the functionality of the basic open-source version of Dovecot. * Storing emails to (high-latency) object storage, initially supporting Amazon S3, Caringo CAStor and Scality. * Email archive storage. See http://www.dovecot.fi/products/index.html for details. I’ve been developing Dovecot for over 10 years now. For a long time it was my primary motivation in life to create the best IMAP server available :) I think I've pretty much accomplished that by now. The future is looking very bright for Dovecot: we will continue the open source development stronger than ever, but in addition, for the long term it needs some additional licensed components that bring the money to cover the cost for future Dovecot development and to be able to build up the support in a professional way. These new features will be added as plugins on top of Dovecot to extend the functionality. Note that I’m not just randomly choosing which features will be open and which will be licensed. Only some specific features will be licensed where my company is going to make money with partnerships and in other measurable ways.

5 6

[Dovecot] mbox vs. maildir storage block waste
by Christoph Anton Mitterer 13 Nov '12

13 Nov '12

Hi. I recently mentioned in several posts, that I'd tended to use mbox rather than maildir, because you don't loose so much space (due to always allocating full blocks per maildir file and thus per mail). I made some tests of my archive, which consists of some 3,4 million mails at a total of 42GB). Most of these mails are probably normal sized, but there are also some with bigger attachments. For those who are interested here are the results: I used a 53687091200 B image file (via loop device) and tested ext4 only. btrfs is IMHO not yet ready, I have had often issues with XFS (corruptions), reiser4 is more or less dead and reiser3 is said to have issues (see e.g. its wikipedia article, even though it has that mode for small files which would fit nicely). As you see the number of mails increased a bit, cause I tested over several days... but this is only a very small increase so it shouldn't change the numbers a lot. 1) Original mbox archives (right now in Evolution) mbox exact space: 38122676224 (does not include meta-data) mbox guess space: 44625670144 (includes Evolution meta-data which is several GBs) mbox num mails: 3412999 (occurances of From_ lines) In the following: - image file, 1B-blocks, Used_begin, Used_end, Available_begin, Available_end result out of df -B 1 - mdir exact used space is the sum of du -B 1 for each regular file (i.e. each mdir file) - mdir guess used space du -B 1 on the root dir of the filesystem - mdir num mails: find . type -f | wc -l on the root dir of the filesystem 2) EXT4 with 4096 blocks: image file: 53687091200 1B-blocks: 52844687360 Used_begin: 188555264 Used_end: 45198778368 Available_begin: 49971777536 Available_end: 2444972032 mdir exact used space: 44810866688 mdir guess used space: 45010243584 mdir num mails: 3423296 delta: 6.688190464 G delta / mail: 1953 B 3) EXT4 with 2048 blocks: image file: 53687091200 1B-blocks: 50324295680 Used_begin: 82857984 Used_end: 41598846976 Available_begin: 47557083136 Available_end: 6041094144 mdir exact used space: 41323991040 mdir guess used space: 41516007424 mdir num mails: 3425033 delta: 3.201314816 G delta / mail: 934 B 4) EXT4 with 1024 blocks: image file: 53687091200 1B-blocks: 50314834944 Used_begin: 38287360 Used_end: 39909360640 Available_begin: 47592193024 Available_end: 7721119744 mdir exact used space: 39683908608 mdir guess used space: 39871086592 mdir num mails: 3425033 delta: 1.561232384 G delta / mail: 455 B As you can see, the delta per mail is rather close to the statistically expected values of 2048B, 1024B and 512B. In the end I probably changed my opinion. ~7GB of wasted block space for all my mails is actually quite a lot, but in days of cheap disk space it's acceptable. And with mbox one has IMHO the major disadvantage that mailservers (including dovecot) store some meta-data _in_ it (i.e. in the mails themselves) , which I don't like a lot. I still think about reports that mbox is much faster with full text search (which sounds reasonable)... but therefore one needs probably and database backend anyway. HTH, Chris.

7 21

[Dovecot] Invalid Managesieve commands are counted twice
by Christoph Bußenius 12 Nov '12

12 Nov '12

Hi, the Managesieve server closes the connection if it receives an unknown command before authentication: "IMPLEMENTATION" "Dovecot Pigeonhole" "SIEVE" "fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave" "NOTIFY" "mailto" "SASL" "PLAIN" "STARTTLS" "VERSION" "1.0" OK "Dovecot ready." --> BOGUS NO "Error in MANAGESIEVE command received by server." NO "Error in MANAGESIEVE command received by server." BYE "Too many invalid MANAGESIEVE commands." Connection closed by foreign host. Note that only one bogus command has been sent by the client, however the server sends two identical error messages. This seems to be a bug in Pigeonhole 0.3.3. In version 0.2.6, the connection was kept open after the error message. This is actually important to us because we use the "sieveshell" utility which is shipped with the Python "managesieve" package. The managesieve.py module always sends a BOGUS command after the TLS handshake. According to its comments, this is done to work around problems with other server implementations: # Some servers send capabilities after TLS handshake, some # do not. We send a bogus command, and expect a NO. If you # get something else instead, read the extra NO to clear # the buffer. typ, data = self._command('BOGUS') (The full source is at http://pydoc.net/managesieve/0.4.2/managesieve) As a result, sieveshell cannot be used with TLS and a current Dovecot/Pigeonhole server. Cheers, Christoph -- Christoph Bußenius Rechnerbetriebsgruppe der Fakultäten Informatik und Mathematik Technische Universität München +49 89-289-18519 <> Raum 00.05.040 <> Boltzmannstr. 3 <> Garching

2 2

[Dovecot] Help me with IMAP config
by Tibby 12 Nov '12

12 Nov '12

Hello! I have dovecot running. Imap works fine. When i connect with outlook an when i delete a message it gets crossed out but still stays in my mailbox. I want to disable this feature. I want it once its deleted then go to trash folder on the mail server and thats it. Is there an option for this? I'm running dovecot 1.2.15 Thanks!

2 1

[Dovecot] Sieve users script problem.
by Eliezer Croitoru 12 Nov '12

12 Nov '12

I am using Dovecot 2.1.9 which seems to solve older bug in sieve. Now I have another problem with users scripts. I want dovecot to run users scripts but it seems to not even see them. Dovecot runs the default sieve script from home dir but wont see the sieve scripts in the sieve subdirectory and some logs: Nov 10 18:35:54 lda(user(a)domain.local): Debug: Quota root: name=User quota backend=maildir args= Nov 10 18:35:54 lda(user(a)domain.local): Debug: Quota rule: root=User quota mailbox=* bytes=1073741824 messages=0 Nov 10 18:35:54 lda(user(a)domain.local): Debug: Quota rule: root=User quota mailbox=Trash bytes=+32212254 (3%) messages=0 Nov 10 18:35:54 lda(user(a)domain.local): Debug: Quota warning: bytes=1020054732 (95%) messages=0 reverse=no command=quota-warning 95 raw mail user Nov 10 18:35:54 lda(user(a)domain.local): Debug: Quota warning: bytes=214748364 (20%) messages=0 reverse=no command=quota-warning 20 raw mail user Nov 10 18:35:54 lda(user(a)domain.local): Debug: none: root=, index=, control=, inbox=, alt= Nov 10 18:35:54 lda(user(a)domain.local): Debug: Destination address: eliezer(a)domain.local (source: user@hostname) Nov 10 18:35:54 lda(user(a)domain.local): Debug: sieve: include: sieve_global_dir is not set; it is currently not possible to include `:global' scripts. Nov 10 18:35:54 lda(user(a)domain.local): Debug: sieve: using the following location for user's Sieve script: /home/vmail/domain.local/eliezer/home/.dovecot.sieve Nov 10 18:35:54 lda(user(a)domain.local): Debug: sieve: opening script /home/vmail/domain.local/eliezer/home/.dovecot.sieve Nov 10 18:35:54 lda(user(a)domain.local): Debug: sieve: script binary /home/vmail/domain.local/eliezer/home/.dovecot.svbin successfully loaded Nov 10 18:35:54 lda(user(a)domain.local): Debug: sieve: binary save: not saving binary /home/vmail/domain.local/eliezer/home/.dovecot.svbin, because it is already stored Nov 10 18:35:54 lda(user(a)domain.local): Debug: sieve: executing script from /home/vmail/domain.local/eliezer/home/.dovecot.svbin Nov 10 18:35:54 lda(user(a)domain.local): Info: sieve: msgid=<XXXX@XXXX>: stored mail into mailbox 'INBOX' So it recognizes the home sir but wont look at the sieve sub directory. My dovecot -n output: # 2.1.9: /etc/dovecot/dovecot.conf # OS: Linux 3.3.8-gentoo x86_64 Gentoo Base System release 2.0.3 ext3 auth_mechanisms = plain login dict { quota = mysql:/etc/dovecot/dovecot-dict-sql.conf.ext } disable_plaintext_auth = no first_valid_uid = 5000 last_valid_uid = 5000 log_path = /var/log/dovecot.log mail_debug = yes mail_gid = 5000 mail_location = maildir:/home/vmail/%d/%n/Maildir/ mail_plugins = " quota" mail_privileged_group = vmail mail_uid = 5000 namespace { inbox = yes list = yes location = prefix = separator = . subscriptions = yes type = private } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=+3%% quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=20%% quota-warning 20 %u sieve = ~/.dovecot.sieve sieve_dir = ~/sieve sieve_extensions = +include +vnd.dovecot.debug +imapflags +spamtest +spamtestplus +relational +comparator-i;ascii-numeric } postmaster_address = postmastert(a)domain.local service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { group = vmail mode = 0600 user = vmail } } service dict { unix_listener dict { group = vmail mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-warning { executable = script /etc/dovecot/quota-warning.sh user = vmail } ssl_ca = </etc/ssl/dca.pem ssl_cert = </etc/ssl/ssl.crt ssl_key = </etc/ssl/ssl.key userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } protocol lda { mail_plugins = " quota sieve" } protocol imap { mail_plugins = " quota imap_quota" } protocol pop3 { pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_uidl_format = %08Xu%08Xv } Thanks, Eliezer

2 2

[Dovecot] Solr 4.0 - lucene - FTS
by Charles Marcus 12 Nov '12

12 Nov '12

Hi Timo, As one who is interested in implementing FTS sometime in the future, I'm curious about what is in store as far as improvements go... Specifically, any plans for implementing immediate/automatic index updates at delivery time? The lack of automatically updated indexes is one downside for its implementation... Also, does the release of Solr 4.0 mean anything for the lucene library used by dovecot? http://www.marketwatch.com/story/lucidworks-congratulates-apache-foundation… Thanks, -- Best regards, Charles

3 6