- dovecot - dovecot.org

result of uid sort by subject
by TACHIBANA Masashi 02 Aug '17

02 Aug '17

Result of uid sort by subject is not expected. for example: Japanese => English => Japanese => English I'm now using Dovecot 2.2.31. command example: HNKK6 UID SORT (REVERSE SUBJECT) utf-7 ALL * SORT 3 1 7 2 4 10 11 8 12 HNKK6 OK Sort completed (0.002 + 0.041 + 0.001 secs). Any hint? -- Masashi Astro TACHIBANA QUALITIA CO., LTD. mailto:tachibana@qualitia.co.jp

2 2

Corrupt index files
by Bruce Guenter 02 Aug '17

02 Aug '17

I am running Dovecot IMAP on Linux, on a LizardFS storage cluster with Maildir storage. This has worked well for most of the accounts for several months. However in the last couple of weeks we are seeing increasing errors regarding corrupted index files. Some of the accounts affected are unable to retrieve messages due to timeouts. It appeared the problems were due to the accounts being accessed from multiple servers simultaneously, so I forced them all to access one server, but the errors remained. It looks like it has something to do with file locking, but LizardFS supports advisory file locking and I do have it enabled. Deleting the corrupted indexes fixes the problem for a while, but it eventually returns, particularly for some accounts. Here are some errors I'm seeing (just a random grab). Actual home directories are munged for confidentiality. imap[25157]: (clientes.standby) Error: Failed to fix view for HOME/clientes:standby/dovecot.index: Missing middle file seq=1 (between 1..1, we have seqs 8): File is already open imap[5565]: (stadiumchair) Error: Transaction log file HOME/stadiumchair/.Drafts/dovecot.index.log: marked corrupted imap[5005]: (stadiumchair) Error: Corrupted transaction log file HOME/stadiumchair/.Drafts/dovecot.index.log seq 2: indexid changed 1418941056 -> 1500658549 (sync_offset=0) imap[20243]: (martha) Error: Transaction log HOME/martha/dovecot.index.log: duplicate transaction log sequence (539) imap[4665]: (emsspam) Error: Index file HOME/emsspam/dovecot.index: indexid changed: 1500658479 -> 1297175382 imap[4665]: (emsspam) Error: Corrupted transaction log file HOME/emsspam/dovecot.index.log seq 3: indexid changed: 1500658479 -> 1297175382 (sync_offset=316) imap[22985]: (emsspam) Error: Corrupted transaction log file HOME/emsspam/dovecot.index.log seq 10742: Invalid transaction log size (9296 vs 9296): HOME/emsspam/dovecot.index.log (sync_offset=9296) imap[3267]: (emsspam) Error: Failed to map view for HOME/emsspam/dovecot.index: Failed to map file seq=10742 offset=9052..18446744073709551615 (ret=0): corrupted, indexid=0 imap[3267]: (emsspam) Error: HOME/emsspam/dovecot.index view is inconsistent: uid=3062271 inserted in the middle of mailbox The output of dovecot -n is pasted in below. Note that some of the boxes are running 4.9, some running 4.4, all have the same problems. Also note that I am using a custom authentication front end for our virtual mailboxes, but it just sets up the minimal environment variables and runs imap. Is there anything I can change to eliminate these problems? Are there any other diagnostics I can provide to shed light on this? # 2.2.31 (65cde28): /etc/dovecot/dovecot.conf # OS: Linux 4.4.66 x86_64 Gentoo Base System release 2.3 log_path = /dev/stderr mail_debug = yes mail_fsync = always mail_location = maildir:~/.maildir mail_log_prefix = "%s[%p]: (%u) " mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = INBOX separator = type = private } passdb { args = * driver = pam } passdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename } ssl_cert = </etc/ssl/dovecot/server.pem ssl_key = # hidden, use -P to show it userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-sql.conf.ext driver = sql } -- Bruce Guenter <bruce(a)untroubled.org> http://untroubled.org/

5 11

under another kind of attack
by Olaf Hopp 31 Jul '17

31 Jul '17

Hi folks, "somehow" similar to the thread "under some kind oof attack" started by "MJ": I have dovecot shielded by fail2ban which works fine. But since a few days I see many many IPs per day knocking on my doors with wron password and/or users. But the rate at which they are knocking is very very low. So fail2ban will never catch them. For example one IP: Jul 25 14:03:17 irams1 dovecot: auth-worker(2212): pam(eurodisc,101.231.247.210,<gAulHSNVsNZl5/fS>): unknown user Jul 25 15:16:36 irams1 dovecot: auth-worker(11047): pam(gergei,101.231.247.210,<dPzYIyRVtOpl5/fS>): pam_authenticate() failed: Authentication failure (password mismatch?) Jul 25 16:08:51 irams1 dovecot: auth-worker(3379): pam(icpe,101.231.247.210,<Ws6t3iRVkOhl5/fS>): unknown user Jul 25 16:10:47 irams1 dovecot: auth-worker(4250): pam(endsulei,101.231.247.210,<dceL5SRVGZVl5/fS>): unknown user Note the timestamps. If I look the other way round (tries to one account) I'll get Jul 25 01:30:48 irams1 dovecot: auth-worker(11276): pam(endsulei,60.166.12.117,<slp6mhhViI48pgx1>): unknown user Jul 25 01:31:26 irams1 dovecot: auth-worker(11276): pam(endsulei,222.243.211.200,<s0+6nBhVabHe89PI>): unknown user Jul 25 13:29:22 irams1 dovecot: auth-worker(4745): pam(endsulei,60.2.50.114,<4elhpCJVtcw8AjJy>): unknown user Jul 25 13:30:27 irams1 dovecot: auth-worker(4747): pam(endsulei,222.84.118.83,<kaE1qCJVn7neVHZT>): unknown user Jul 25 16:10:47 irams1 dovecot: auth-worker(4250): pam(endsulei,101.231.247.210,<dceL5SRVGZVl5/fS>): unknown user Jul 25 16:11:45 irams1 dovecot: auth-worker(5933): pam(endsulei,206.214.0.120,<R5H56CRVdJfO1gB4>): unknown user Also note the timestamps! And I see many many distinct IPs per day (a few hundred) trying many many existing and non-existings accounts. As you see in the timestamps in my examples, this can not be handled by fail2ban without affecting regular users with typos. Is anybody observing something similar ? Anybody an idea against this ? Many of these observed IPs are chinese mobile IPs, if this matters. But we have also chinese students and researchers all abroad. Regards, Olaf -- Karlsruher Institut für Technologie (KIT) ATIS - Abt. Technische Infrastruktur, Fakultät für Informatik Dipl.-Geophys. Olaf Hopp - Leitung IT-Dienste - Am Fasanengarten 5, Gebäude 50.34, Raum 009 76131 Karlsruhe Telefon: +49 721 608-43973 Fax: +49 721 608-46699 E-Mail: Olaf.Hopp(a)kit.edu atis.informatik.kit.edu www.kit.edu KIT – Die Forschungsuniversität in der Helmholtz-Gemeinschaft Das KIT ist seit 2010 als familiengerechte Hochschule zertifiziert.

11 16

Re: under another kind of attack
by Davide Marchi 31 Jul '17

31 Jul '17

> From: Olaf Hopp <Olaf.Hopp(a)kit.edu> > Davide, > yours is all postfix and thus has got no overlap with dovecot. > So no interference. > Olaf Yes, I know, but I preferred not to give anything for granted ;-) Many Thanks Olaf!

1 0

Large Scale Cluster using dsync and tagged director
by Matt Bryant 31 Jul '17

31 Jul '17

Just been reading up on some of dovecot features and was wondering if anyone had played around with large scale systems made up from mutliple small dsync clusters and using tagged director to split users between nodes of a sync'd cluster (rather than nfs). I know traditionally its been shared storage and multiple nodes and dovecots own doco suggest dsync for several thousands rather than millions but given the cloud seems to be all about replication what about many smaller nodes using dsync and scaling horizontally ? The only issue is how does director scale when there are serveral(or rather lots of) 'tags' ... in fact what are the limitations with director ? Matt -- <https://www.postbox-inc.com/?utm_source=email&utm_medium=siglink&utm_campai…>

1 0

dovecot auth errors for a new user
by Ruben Safir 31 Jul '17

31 Jul '17

I've been running dovecott without trouble for quite a why and now when I added a new user, it is not accepting the user and I can not track the problem. It says find more information in the server log, but it is not in /var/log/messages or /var/log/mail.err and nothing with lsof dovecot|grep log show anything to tail www:~ # dovecot -n # 2.2.13: /etc/dovecot/dovecot.conf # OS: Linux 3.16.7-53-pae i686 openSUSE 13.2 (i586) base_dir = /var/run/dovecot/ managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } plugin { sieve = ~/.dovecot.sieve sieve_dir = ~/sieve } protocols = pop3 ssl_cert = </etc/ssl/certs/dovecot.pem ssl_key = </etc/ssl/private/dovecot.key userdb { driver = passwd } www:/etc/dovecot # dovecot --version 2.2.13 Sending of password for user facebook did not succeed. Mail server mrbrklyn.com responded: Internal error occurred. Refer to server log for more information. 2017-07-30T15:41:58.803006-04:00 www dovecot: pop3-login: Login: user=<facebook>, method=PLAIN, rip=10.0.0.62, lip=96.57.23.82, mpid=25269, TLS, session=<D5J/G45VQgAKAAA+> 2017-07-30T15:41:58.812827-04:00 www dovecot: pop3(facebook): Error: user facebook: Initialization failed: Namespace '': Mail storage autodetection failed with home=/home/facebook 2017-07-30T15:41:58.816903-04:00 www dovecot: pop3(facebook): Error: Invalid user settings. Refer to server log for more information. -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www2.mrbrklyn.com/resources - Unpublished Archive http://www.coinhangout.com - coins! http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013

2 4

Re: under another kind of attack
by Davide Marchi 30 Jul '17

30 Jul '17

Hi to all, @Olaf Hopp I've this filter enabled for fail2ban, my question is: could my filters overlap or interfere with those suggested by you? this is my filter: Contents of /etc/fail2ban/jail.conf: [postfix] # Ban for 10 minutes if it fails 6 times within 10 minutes enabled = true port = smtp,ssmtp filter = postfix logpath = /var/log/mail.log maxretry = 6 bantime = 600 findtime = 600 Contents of /etc/fail2ban/filter.d/postfix.conf: # Fail2Ban configuration file # Author: Cyril Jaquier # $Revision$ [Definition] # Option: failregex # Notes.: regex to match the password failures messages in the logfile. The # host must be matched by a group named "host". The tag "<HOST>" can # be used for standard IP/hostname matching and is only an alias for # (?:::f{4,6}:)?(?P<host>[\w\-.^_]+) # Values: TEXT # # Jul 11 02:35:08 mail postfix/smtpd[16299]: lost connection after AUTH from unknown[196.12.178.73] failregex = lost connection after AUTH from unknown\[<HOST>\] # Option: ignoreregex # Notes.: regex to ignore. If this regex matches, the line is ignored. # Values: TEXT # ignoreregex = Many thanks!

2 1

Re: under another kind of attack
by Joseph Tam 28 Jul '17

28 Jul '17

(I think I am testing other readers' patience, so if you want to follow-up, you can Email me directly.) > but how often do you have to type your username ? Not often, but I'm not talking the typical case. The larger the population you serve, the more circumstances you'll have to cover. > Only on the initial config of your mailer. After that you are done. Mail reader setups by users is often an error prone process, judging from the number of times I have to correct a setup. This is especially true for an educational institution that typically has a large turnover of accounts. If a user gets it initially wrong, then fixes their mistake, they can't get it to work despite trying all sorts of config variations, not realizing it can't be resolved anymore. Result: trouble call. If someone blows it setting on a multi-user workstation, other users with a working setups can't log in. Result: trouble call. If a student flubs their credentials, all the roommates behind their residential NAT gateway suffer. Result: trouble call. If a user screws up using an external service that slurps their mail (e.g. Gmail, Yahoo, uniboxapp, etc.), or worse, someone malicious does it on puropse, all other users of this service will be DoS'd. Result: trouble call(s). If a user acquires a DHCP address in a polluted network ..., well you get the idea. Not to mention ex-users who forget to remove their mail accounts from their smartphones, leaving a trail of blacklist entries in their wake as they travel from coffee shops to other public WiFi hotspots. > So this is why I decided to use two distinct jails with > different policies. It seems to work reasonable well. Until it doesn't. If it works for you, more power to you. The cost/benefit of a hair-trigger blacklist policy is that it saves you a few log entries showing futile attempts at finding weak passwords (because you have strong passwords, don't you?) at the risk of dealing with any of the above situations. Joseph Tam <jtam.home(a)gmail.com>

1 0

Re: under another kind of attack
by Joseph Tam 27 Jul '17

27 Jul '17

Olaf Hopp <Olaf.Hopp(a)kit.edu> wrote: > And I have a new one just for "unknown user" and here my bantime and findtime > are much bigger and the retries are just '2'. So here I'm much harsher. > I'll keep an eye on my logs and maybe some more twaeking is necessary. Just be careful about typos (like twaeking!): users could simply misspell their username, or get mixed up with some another account or alias. This is why I favour targetting known bad accounts, not merely accounts that don't exist. Joseph Tam <jtam.home(a)gmail.com>

2 1

Re: failed to store into mailbox 'INBOX/Junk': Permission
by Davide Marchi 26 Jul '17

26 Jul '17

> From: Thomas Leuxner <tlx(a)leuxner.net> >> cat /etc/dovecot/dovecot-acl >> * user=booking(a)hotelsangiorgioriccione.com lrwsi >> * user=info(a)hotelsangiorgioriccione.com lrwsi > Hi Davide, > > For LMTP to file the mails you need to add the p flag (POST). Well, I've added the "p" flag and now I will monitor the situation ;-) PS Just as an additional note, just after making this change and reload Dovecot I have detected (oneshot) this error message: Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF Jul 26 22:09:53 server dovecot: auth: Error: read(anvil-auth-penalty) failed: EOF Jul 26 22:09:53 server dovecot: auth: Error: net_connect_unix(anvil-auth-penalty) failed: Permission denied Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF Jul 26 22:09:53 server dovecot: imap-login: Error: read(anvil) failed: EOF But now everything seems to work well (let's hope) ;-) Many thanks Thomas!

1 0