- dovecot - dovecot.org

Re: [Dovecot] pop3 rate limit
by michael 30 Dec '13

30 Dec '13

On 2013-12-29 02:05, Christian Rößner wrote: > Hi, > > we have customers with Exchange servers that are polling for new mail > every minute with dozens of pop3 accounts. I am looking for a > mechanism to rate limit this per user. So what I am looking for is a > way to block users from polling, if a user asks for new mail more > than > every 5 minutes (for example). > > Is this possible? Can this be achieved within Dovecot or does it need > external scripting? (I thought about fail2ban, but also want IPv6 > support) > > Thanks in advance > > -Christian Rößner See: www.policyd.org You'll need to use v2.1 to get IPV6 support. Michael Hallager

6 10

[Dovecot] Warning: fscking index file /.../dovecot.index
by Alex Vitola 30 Dec '13

30 Dec '13

We have often found the below error in VPS servers using OpenVZ, and some cases using OpenStack, in neither case is used for storage or sharing ISCSI or NFS Client can not receive messages via POP3 or IMAP In /var/log/maillog the message below is seen Oct 31 15:28:26 vps dovecot: pop3(user(a)domain.com): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0 [0x2b3718e07400] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2b3718e07456] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2b3718e06df3] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b3718b6566b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0x89) [0x2b3718b5ced9] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b3718aeae4d] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b3718aea357] -> /usr/lib64/dovecot/libdovecot-storage.so.0 [0x2b3718aea698] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xbb) [0x2b3718aea83b] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x31) [0x2b3718b15e31] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x2b3718b18537] -> dovecot/pop3(client_create+0x332) [0x405702] -> dovecot/pop3 [0x40414b] -> dovecot/pop3 [0x404708] -> /usr/lib64/dovecot/libdovecot.so.0 [0x2b3718dc9a52] -> /usr/lib Both POP3 as IMAP Oct 31 15:24:42 vps dovecot: imap(user(a)domain.com): Warning: fscking index file /home/user/mail/domain.com/user/dovecot.index Oct 31 15:28:26 vps dovecot: pop3(user(a)domain.com): Warning: fscking index file /home/user/mail/domain.com/user/dovecot.index Today I do to solve this the procedure below - Access the directory mailbox - Remove files dovecot.index * - Run the command doveadm index-u $ user @ $ domain-q INBOX The question is for what reason this only occurs with VPS servers, I do not find this error on any single physical server to virtual and dovecot settings are the same. att Alex Vitola

1 0

[Dovecot] dovecot 2.2.10 - imap: Error: Can't load plugin imap_stats_plugin:
by giovanni＠neweden.uk.to 30 Dec '13

30 Dec '13

dovecot 2.2.10 is raising this error on my installation. Dec 30 13:57:59 imap: Error: Can't load plugin imap_stats_plugin: Plugin stats must be loaded also (you must set: mail_plugins=$mail_plugins stats) Dec 30 13:57:59 imap: Fatal: Couldn't load required plugins This error occurs at logging phase. The dovecot configuration looks correct to me, here below an excerpt of it: ... ... mail_plugins=$mail_plugins stats protocol imap { mail_plugins = $mail_plugins imap_stats } plugin { # how often to session statistics stats_refresh = 30 secs # track per-IMAP command statistics stats_track_cmds = yes } ... ... I was already using this configuration but only now I am getting this error. From STRACE I see that the stat plugin is correctly localed . Is anybody else getting this error? or am I missing something? Here below an except of the strace message. [pid 17112] open("/usr/local/lib/dovecot/lib95_imap_stats_plugin.so", O_RDONLY) = 14 [pid 17112] read(14, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\t\0\0004\0\0\0"..., 512) = 512 [pid 17112] fstat64(14, {st_mode=S_IFREG|0755, st_size=53122, ...}) = 0 [pid 17112] mmap2(NULL, 8908, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 14, 0) = 0xf7609000 [pid 17112] mmap2(0xf760b000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 14, 0x1) = 0xf760b000 [pid 17112] close(14) = 0 [pid 17112] open("/usr/local/lib/dovecot/lib90_stats_plugin.so", O_RDONLY) = 14 [pid 17112] read(14, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\24\0\0004\0\0\0"..., 512) = 512 [pid 17112] fstat64(14, {st_mode=S_IFREG|0755, st_size=100054, ...}) = 0 [pid 17112] mmap2(NULL, 20320, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 14, 0) = 0xf7604000 [pid 17112] mmap2(0xf7608000, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 14, 0x3) = 0xf7608000 [pid 17112] close(14) = 0 [pid 17112] write(2, "\1\01017112 prefix=imap: \n", 22) = 22 [pid 17112] write(2, "\1\00417112 Can't load plugin imap_s"..., 127) = 127 [pid 17112] munmap(0xf7609000, 8908) = 0 [pid 17112] munmap(0xf7604000, 20320) = 0 [pid 17112] write(2, "\1\00517112 Couldn't load required p"..., 39) = 39 [pid 17112] exit_group(89) = ? Process 17112 detached

2 2

[Dovecot] dsync-local crashes due to assertion failures
by Fabian Groffen 30 Dec '13

30 Dec '13

Hi, I have two servers that share nothing, running dovecot version 2.2.9. The first server hosts the original mail base, which I want to sync using dovecot to the second server. With some trial and error I managed to get synchronisation running using tcp sockets. Now for some accounts I see dsync-local crashing on the server that has the original mail base (which is still the only of the two that receives new mail). The log shows: Dec 8 03:13:14 zeus dovecot: dsync-local(someuser): Panic: file mail-index-tr ansaction-export.c: line 203 (log_append_ext_hdr_update): assertion failed: (u32 .offset + u32.size <= ext_hdr_size) Dec 8 03:13:14 zeus dovecot: dsync-local(someuser): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0x70ea0) [0x7f3ee4137ea0] -> /usr/lib64/dovecot/libdovecot.so.0(+0x70efe) [0x7f3ee4137efe] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7f3ee40ebd2c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_export+0xbed) [0x7f3ee442e89d] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x89104) [0x7f3ee442d104] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit_full+0xb2) [0x7f3ee442d592] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_transaction_commit+0xc) [0x7f3ee442d63c] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mail_index_sync_commit+0xb6) [0x7f3ee44372b6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x335d6) [0x7f3ee43d75d6] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x328bc) [0x7f3ee43d68bc] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x32d83) [0x7f3ee43d6d83] -> /usr/lib64/dovecot/libdovecot-storage.so.0(maildir_storage_sync_init+0xd9) [0x7f3ee43d7199] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync_init+0x2e) [0x7f3ee43e7f2e] -> /usr/lib64/dovecot/libdovecot-storage.so.0(mailbox_sync+0x27) [0x7f3ee43e8047] -> dovecot/doveadm-server(dsync_brain_master_send_mailbox+0xcf) [0x41f17f] -> dovecot/doveadm-server(dsync_brain_run+0x2c8) [0x41d738] -> dovecot/doveadm-server() [0x41dd70] -> dovecot/doveadm-server() [0x42eda0] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f3ee41495a6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xbd) [0x7f3ee414a62d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3ee4149008] -> dovecot/doveadm-server() [0x41bb6c] -> dovecot/doveadm-server() [0x40ee3b] -> dovecot/doveadm-server() [0x419581] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_call_io+0x36) [0x7f3ee41495a6] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_handler_run+0xbd) [0x7f3ee414a62d] -> /usr/lib64/dovecot/libdovecot.so.0(io_loop_run+0x38) [0x7f3ee4149008] -> /usr/lib64/dovecot/libdovecot.so.0(master_service_run+0x13) [0x7f3ee40f1643] Dec 8 03:13:14 zeus dovecot: dsync-local(someuser): Fatal: master: service(doveadm): child 14116 killed with signal 6 (core dumps disabled) It seems that despite these crashes, some data is being transferred. Newer accounts seem not to have problems, this happens with older accounts only, for as far as I can see. I tried removing the index files, but that didn't seem to help. Any ideas on what I can do here? Both servers are configured to use maildir storage. Thanks, Fabian -- Fabian Groffen Gentoo on a different level

1 1

[Dovecot] pop3 rate limit
by Christian Rößner 30 Dec '13

30 Dec '13

Hi, we have customers with Exchange servers that are polling for new mail every minute with dozens of pop3 accounts. I am looking for a mechanism to rate limit this per user. So what I am looking for is a way to block users from polling, if a user asks for new mail more than every 5 minutes (for example). Is this possible? Can this be achieved within Dovecot or does it need external scripting? (I thought about fail2ban, but also want IPv6 support) Thanks in advance -Christian Rößner -- [*] sys4 AG http://sys4.de, +49 (89) 30 90 46 64 Franziskanerstraße 15, 81669 München Sitz der Gesellschaft: München, Amtsgericht München: HRB 199263 Vorstand: Patrick Ben Koetter, Axel von der Ohe, Marc Schiffbauer Aufsichtsratsvorsitzender: Florian Kirstein

2 1

[Dovecot] Sieve's spamtest always returns 0
by martin f krafft 28 Dec '13

28 Dec '13

I am a bit at a loss here with Sieve (pigeonhole) and the spamtest extension. I am using Dovecot 2.1.7 (backported to Debian squeeze), which comes with Pigeonhole 0.3.0. Messages are scanned with SpamAssassin, which adds a header like X-Spam-Status: Yes, score=84.6 required=5.0 tests=… and so I configured spamtest in conf.d/90-plugin.conf like so: sieve_spamtest_status_type = score sieve_spamtest_status_header = X-Spam-Status: [^,]*, score=(-?[[:digit:]]+\.[[:digit:]]).* sieve_spamtest_max_header = X-Spam-Status: [^,]*, score=[^[:space:]]+ required=(-?[[:digit:]]+\.[[:digit:]]).* I tested those regular expressions with sed -r, e.g. % sed -rne "s@^X-Spam-Status: [^,]*, score=-?[[:digit:]]+\.[[:digit:]] required=(-?[[:digit:]]+\.[[:digit:]]).*@\1@p" mailfile 5.0 and they work. Unfortunately, in sieve scripts, the spamtest value is always 0, which is indicative of the spamtest "not having run", which in this case I assume means that the regular expression didn't match. Am I right in assuming that the matching happens at the time of evaluation, and so adding the headers using vnd.dovecot.filter just before works? Or does the spamtest matching happen before the sieve scripts are executed? Can you see any other reason why spamtest always yields a value of 0? Thanks, -- martin | http://madduck.net/ | http://two.sentenc.es/ "those who are faithful know only the trivial side of love: it is the faithless who know love's tragedies." -- oscar wilde spamtraps: madduck.bogus(a)madduck.net

3 4

[Dovecot] Dovecot 'Submission' service?
by Charles Marcus 27 Dec '13

27 Dec '13

I do recall this being discussed, but I can't remember if it is actually in a working state now or not. According to the wiki, it is a work in progress: http://wiki2.dovecot.org/Roadmap?highlight=%28submission%29 So - is it in a functional state yet? Thanks, -- Best regards, */Charless/***

2 2

[Dovecot] Using DSync to convert from maildir to m/sdbox during migration of mail to new server
by Charles Marcus 27 Dec '13

27 Dec '13

Ok, now to work out the precise method (config for the target server and dsync command) to use to perform the migration from the old server to the new one, converting to dbox (s or m, depending on how the other thread discussing this turns out). I'll update the old server to 2.2.10 tomorrow morning so they're both at the same version. So, assuming both servers are running 2.2.10, with OldServer: mail_location = maildir:~/:INDEX=MEMORY and NewServer: mail_location = sdbox:~/sdbox or mail_location = mdbox:~/mdbox After reading and re-reading: http://wiki2.dovecot.org/Tools/Dsync I'm not so sure... First problem - apparently it can only be used one user at a time? And - do you have to actually know the users password too? If so, then this probably isn't going to be easier, and maybe even harder than just rsyncing everything, then slowly converting each user one at a time. -- Best regards, */Charles/*

1 0

[Dovecot] Migrating from Maildir to dbox+SIS... sdbox or mdbox?
by Charles Marcus 27 Dec '13

27 Dec '13

Ok... Now that I've got the niggling issues resolved, on to bigger and better things. I want to convert everyone from Maildir to dbox and enable SIS (our users use a lot of large attachments), and after some reading/googling, I think it may be better to use dsync instead of rsync for the actual migration after all since it will do this at the same time as the syncing. If I'm not mistaken, once I've figured out the correct syntax for the command, I should be able to do the conversion and migration at the same time. So, that is what I want to do, but I still have to choose between sdbox and mdbox. I tend to feel a little safer with sdbox, just because it seems more stable - and I like the idea that if a file gets corrupted somehow, then it is only that one mail that may potentially be lost, but if mdbox indexes get corrupted, you are guaranteed to lose messages (maybe a lot)... But I also really like the idea of an order of magnitude fewer actual files (would make rsync based backups much faster, etc). Timo - the last comments I was able to find from you were back in 2011... would you mind commenting on the current stated of dbox - especially mdbox, as far as reliability goes? The 'scary warnings' about how important the index files are ... well ... scares me... ;) I know there were definitely some issues with mdbox a few years ago - I found the old thread from 2011, but nothing substantial since, so... So... are there any people here, especially larger installs, using mdbox? sdbox? I'd especially be interested in hearing from anyone who has used both, and picked one over the other, and their reasons for doing so. Thanks, -- Best regards, */Charles/*

1 0

[Dovecot] Enabling SIS (single instance storage)
by Charles Marcus 27 Dec '13

27 Dec '13

Ok, didn't find much on this... The bottom of /etc/dovecot/conf.d/10-mail.conf has the following settings: #mail_attachment_dir = #mail_attachment_min_size = 128k #mail_attachment_fs = sis posix #mail_attachment_hash = %{sha1} all of which are apparently the defaults according to doveconf -d output - so no need to set them explicitly, unless I want to change them, right? So, enable the first one - but - the comments don't say anything about supporting variables. I'm assuming you can, otherwise all attachments for all domains would be in a single directory, which some admins might not like (I wouldn't)... so hopefully I can just set it to this: mail_attachment_dir = /var/vmail/%d/.attachments So how does this work during conversion? Users still on maildir simply don't engage it? But as users are converted to dbox, all attachments are de-duped, and then new mails coming in with attachments (for converted users) are de-duped at delivery? The wiki could do with a bit more on enabling and implementing it, especially what to expect when enabling it for the first time on a production system and what to expect along the way when converting existing users... -- Best regards, */Charles/*

1 0