- dovecot - dovecot.org

Comparing Indexing Speed + ZFS question
by Qmail 18 Apr '18

18 Apr '18

Hi, have two quick questions: I am re-indexing some large mailboxes and I wanted to get some opinions on these speeds. I ran a random sort command on an unindexed mailbox in order to get it to index. Results, for a mailbox with 10GB Inbox, spread in 65 000 files: 3 UID SORT (ARRIVAL) US-ASCII OR OR OR FROM "dfgfgfgf" SUBJECT "dfgfgfgf" TO "dfgfgfgf" CC "dfgfgfgf" a03 select INBOX * SORT 3 OK Sort completed (2455.549 + 0.122 + 1.477 secs). Index cache file at the end was 31MB in size: 30756376 Apr 18 02:14 dovecot.index.cache Is that normal, slow or fast ? Second question is, how can I speed this up ? I store messages using Director over NFS , on top of ZFS. From what I've read it would be fairly safe to disable mail_fsync but I don't think that would help much with the indexing speed. Any other tips ?

2 2

Postfix sendmail cannot be called from Sieve redirect
by Ulrich Zehl 17 Apr '18

17 Apr '18

Sending (in my case: forwarding) messages from Sieve via sendmail does not work with version 2.3.1. I have narrowed it down to this simple test case: # dovecot -n # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 4.4.0-119-generic x86_64 Ubuntu 16.04.4 LTS # Hostname: tuxi.topfen.net first_valid_gid = 200 first_valid_uid = 200 mail_location = mbox:~/mail passdb { args = password=secret driver = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = lmtp service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } userdb { args = uid=200 gid=200 home=/home/vmail driver = static } protocol lmtp { mail_plugins = sieve } # cat /home/vmail/.dovecot.sieve redirect "test(a)example.invalid"; stop; $ swaks --protocol lmtp -s 127.0.0.1 -f ulrich(a)topfen.net -t ulrich(a)topfen.net === Trying 127.0.0.1:24... === Connected to 127.0.0.1. <- 220 tuxi.topfen.net Dovecot ready. -> LHLO tuxi.topfen.net <- 250-tuxi.topfen.net <- 250-8BITMIME <- 250-CHUNKING <- 250-ENHANCEDSTATUSCODES <- 250-PIPELINING <- 250 VRFY -> MAIL FROM:<ulrich(a)topfen.net> <- 250 2.1.0 OK -> RCPT TO:<ulrich(a)topfen.net> <- 250 2.1.5 OK -> DATA <- 354 OK -> Date: Tue, 17 Apr 2018 18:44:42 +0200 -> To: ulrich(a)topfen.net -> From: ulrich(a)topfen.net -> Subject: test Tue, 17 Apr 2018 18:44:42 +0200 -> X-Mailer: swaks v20130209.0 jetmore.org/john/code/swaks/ -> -> This is a test mailing -> -> . <** 451 4.3.0 <ulrich(a)topfen.net> Temporary internal error -> QUIT <- 221 2.0.0 Bye === Connection closed with remote host. $ tail -n 10 /var/log/mail.log Apr 17 18:45:36 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/421882.22643: Permission denied Apr 17 18:45:36 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/421882.22643: Permission denied Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: program `/usr/sbin/sendmail' was forcibly terminated with signal 15 Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: sieve: msgid=unspecified: failed to redirect message to <test(a)test.at>: Failed to execute sendmail (temporary failure) Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: sieve: Execution of script /home/vmail/.dovecot.sieve was aborted due to temporary failure (user logfile /home/vmail/.dovecot.sieve.log may reveal additional details) Apr 17 18:45:36 tuxi dovecot: lmtp(22639): Disconnect from 127.0.0.1: Client has quit the connection (state = READY) Apr 17 18:45:46 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/422091.22643: Permission denied Apr 17 18:45:46 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/422091.22643: Permission denied Apr 17 18:45:56 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/422227.22643: Permission denied Apr 17 18:45:56 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/422227.22643: Permission denied As far as I can tell, this is because Dovecot somehow does not allow suid programs to run (Postfix' postdrop(1), which is invoked by sendmail(1), requires set-gid to write the queue file). What can I do to change this (besides using SMTP from Dovecot directly)? I am using Ubuntu 16.04 and the Dovecot packages from the community repository: ~$ dpkg -l dovecot* | grep ^ii ii dovecot-core 2:2.3.1-1 amd64 secure POP3/IMAP server - core files ii dovecot-lmtpd 2:2.3.1-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-sieve 2:2.3.1-1 amd64 secure POP3/IMAP server - Sieve filters support Ulrich

2 2

dovecot-lda "-f" argment optional?
by Sebastian Arcus 17 Apr '18

17 Apr '18

I use dovecot-lda to deliver incoming email from Exim to Dovecot. After having some problems with email bounces with empty envelope sender not being delivered because of an empty -f argument, I discovered that Exim can pass email to Dovecot using dovecot-lda without using the -f argument at all. Would this have any adverse effects anywhere else? The docs don't even seem to specify if -f is optional or not - but I guess it is - as it seems to be working without it. Does anybody know what is its role? I'd like to understand things a bit before I just start removing it permanently from my setup.

1 0

[bug] Auth cache, proxy and ITERINDEX
by azurit＠pobox.sk 17 Apr '18

17 Apr '18

Hi, today we have exceprienced really major difficulties with our proxy backend servers. Everything started after enabling auth cache: auth_cache_size = 100M auth_cache_verify_password_with_worker = yes auth_cache_ttl = 1 hour auth_cache_negative_ttl = 1 hour Soon after we start receiving lots of calls from our customers about 'missing all e-mail messages'. This was in errors logs on master server: Apr 16 14:37:10 server00 dovecot: imap(<censored_email_address>): Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/var/mail/vhosts/<censored_domain>/<censored_name>/home/dovecot.autoexpunge.lock) failed: safe_mkstemp(/var/mail/vhosts/<censored_domain>/<censored_name>/home/dovecot.autoexpunge.lock) failed: No such file or directory Looks like that all users, who are placed on proxy backends, were logged correctly BUT not proxied to the right server so they saw empty mailboxes (Dovecot also created some directories on master server). What is worse, their email software deleted all local emails so they are now downloading everything from scratch (100Gs of mails so it will take some time). After auth cache was disabled, things started to go to normal EXCEPT users were reporting they don't see all folders - which was true, only few on them were visible (INBOX, trash, sent etc.). We tried everything to make them visible again, for example deleting indexes, but nothing helped. Finally, removing option ITERINDEX from mail_location helped. Can anyone explain what happened? Thank you azurIt

2 2

spamc scripts in IMAPSieve docs.
by Reio Remma 17 Apr '18

17 Apr '18

Hello! I noticed SpamAssassin *spamc* usage has entered the documentation at https://wiki2.dovecot.org/HowTo/AntispamWithSieve I'm wondering if the -C (report) option in sa-learn-ham.sh should use revoke instead of report for --ham messages? I started using imapsieve with spamc myself just a few weeks ago, but I haven't used the reporting ability yet. Reio

2 4

File system permissions - setgid bit and Netapp NFS volumes
by Shaun Johnson 17 Apr '18

17 Apr '18

Greetings Dovecot List, I have a bit of an edge case I am trying to resolve. I am currently using dovecot on Ubuntu 14.04 - Ubuntu package version: 1:2.2.9-1ubuntu2.3 I have attached the output of doveconf -n to this email - but to describe the configuration in a nutshell: my server is configured to use Maildir storage I do not use dovecot delivery service (there is a separate program in my system delivering messages) I am using a 'checkpassword' mechanism for authentication My mail storage is mounted via NFSv3 from a Netapp Filer FAS3050 configured to set anonuid=89, read write access, and root access. Dovecot has been configured with: mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mmap_disable = yes lock_method = fcntl In order to function with the other components of my system, the Mail Storage directories are expected to have permissions: drwx--S--- uid 89: gid 89 The oddity I am encountering came up recently when I set up a job to monitor and correct directory permissions. I found after observing the system in action that overtime, the ~/Maildir directories for all active mailboxes would mysteriously lose the setgid bit. Now, while the system still functioned, it was something unexpected so I investigated further by turning on auditd to monitor what was changing the permissions on the path. After tracing through the results, I found an entry indicating that the 'imap' process of dovecot was performing a 'chown' call on the directory, with an end result of the previous mode 2700 being changed to 0700. Now I understand that on some filesystems, a 'chown' call can end up losing any setuid or setgid bits on an inode - and apparently Netapp's WAFL file system is one of them. Looking at the dovecot source code, I *believe* the source of the chown call is in src/lib/nfs-workarounds.c function nfs_flush_chown_uid - where a 'chown' call is being used to flush NFS attribute caches? for what I believe is the purposes of ascertaining file lock status? My question for you folks is if anyone else has encountered this on other file systems (or filers) - and if there is a native method in dovecot - either the version I am currently running or an updated newer version - that allows one to either retain the permissions assigned to a directory or specify the permissions I expect on the Maildir directory structures. Any feedback would be welcome, thanks! -- Shaun Johnson LinuxMagic Inc. 604-682-0300 Beautiful British Columbia, Canada

2 2

LDAP Homedir location: Needs dovecot restart after change it
by Andre Luiz Paiz 16 Apr '18

16 Apr '18

Dear group members. I work with Dovecot and Openldap authentication. Sometimes users change departments and we need to alter their homedir location. Every time this process is needed, I perform this steps: 1 - Change homedir location in openldap 2 - Move homedir folder to the new location 3 - Re-apply permissions 4 - Remove user index folder After I do that, users cannot authenticate unless I restart dovecot, process that I would like to avoid. Can you guys give a tip on what I need to change to avoid this last problematic step? After the restart, everything works. Does the auth_cache feature also store the homedir location? System: CentOS 7 Versions: dovecot-pigeonhole-2.2.10-8.el7.x86_64 dovecot-2.2.10-8.el7.x86_64 Doveconf -n: # 2.2.10: /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-693.5.2.el7.x86_64 x86_64 CentOS Linux release 7.4.1708 (Core) auth_cache_negative_ttl = 0 auth_cache_size = 10 M auth_mechanisms = plain login auth_username_format = %Ln default_client_limit = 1024 default_process_limit = 1024 default_vsz_limit = 1 G listen = * mail_location = maildir:~/Maildir:INDEX=/var/indexes/dovecot/%u mail_plugins = " mail_log notify quota" mail_temp_dir = /var/tmp managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date ihave imapflags notify mbox_write_locks = fcntl mmap_disable = yes namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = } passdb { driver = pam } passdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } plugin { mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename mail_log_fields = uid box msgid size quota = maildir quota_exceeded_message = Cota de armazenamento de e-mails excedida. Quota exceeded for mail store. quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=85%% quota-warning 85 %u quota_warning3 = storage=75%% quota-warning 75 %u sieve = ~/.dovecot.sieve sieve_before = /var/lib/dovecot/sieve/default.sieve sieve_default = /var/lib/dovecot/sieve/default.sieve sieve_dir = ~/sieve sieve_extensions = +notify +imapflags } protocols = imap pop3 lmtp sieve service anvil { client_limit = 14100 process_limit = 1 } service auth-worker { client_limit = 1 process_limit = 50 service_count = 1 } service auth { client_limit = 15100 process_limit = 1 } service dict { client_limit = 1 } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_limit = 6000 process_min_avail = 10 service_count = 1 vsz_limit = 256 M } service imap { process_limit = 3000 vsz_limit = 1 G } service lmtp { client_limit = 1 inet_listener lmtp { port = 24 } unix_listener lmtp { mode = 0666 } } service managesieve-login { inet_listener sieve { port = 4190 } } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } process_limit = 6000 process_min_avail = 10 service_count = 1 } service pop3 { process_limit = 3000 } service quota-warning { executable = script /usr/local/bin/quota-warning.sh unix_listener quota-warning { mode = 0666 user = dovecot } user = root } ssl = required ssl_cert = ********** ssl_key = *********** userdb { driver = passwd } userdb { args = /etc/dovecot/dovecot-ldap.conf.ext driver = ldap } protocol lmtp { mail_plugins = " mail_log notify quota sieve quota" } protocol imap { mail_plugins = " mail_log notify quota imap_quota" } Thanks Andre

2 1

Is FTS needed for Subject/Recipient/Sender server-side searches?
by Sebastian Arcus 16 Apr '18

16 Apr '18

I've had a number of Dovecot installs happily running for quite a few years now, but I've just stumbled over a new issue. I am trying to get some iPhones to do server-side email searches through ActiveSync/Exchange protocol -> Horde -> Dovecot. The bit I can't figure out is if I need to enable and configure FTS in Dovecot for that to work? I've tested server-side email searches against the same Dovecot server on Thunderbird - through IMAP (there is an option to force the search on the server in Thunderbird) - and it is working really well and fast. I am only really interested in Sender/Recipient/Subject searches. Server-side searching is not working at the moment on the iPhones side - although I can see the ActiveSync commands being sent to Horde. Anyway, to begin with, does anybody know if server-side IMAP searches need FTS in Dovecot (Horde is talking to Dovecot through IMAP). The fact that they work fine in Thunderbird would suggest the answer is 'no' - but could someone confirm that is correct please? Anticipated thanks for any hints. I am on Dovecot 2.2.33 if in case it matters.

1 0

Migration from Maildir to mdbox on another server preserving POP3 UIDLS
by DurgaPrasad - DatasoftComnet 15 Apr '18

15 Apr '18

Hi all, Does anyone have a good migration document for Migration from Maildir (old server 2.1.x) to mdbox onto another new server with SIS preserving POP3 UIDLS. Preserving UIDLs is of paramount importance so that clients don't download mails again into outlook. Regards/DP --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

1 0

52.184.164.73 in my logs
by daniel_1983＠protonmail.com 15 Apr '18

15 Apr '18

Dear list, One of my users is reading e-mail from his phone. When he logs in, this is what I see in my logs : Apr 10 16:17:58 auth-worker(17101): Debug: sql(xxx(a)mydomain.tld,52.184.164.73): query: SELECT email as user, password FROM users WHERE email = LOWER('xxx(a)mydomain.tld') [...] Apr 10 16:17:58 imap-login: Info: Login: user=<xxx(a)mydomain.tld>, method=LOGIN, rip=52.184.164.73, lip=10.10.10.19, mpid=19286, TLS, session=<E6cEB4BprgA0uKRJ> Both lines show a remote IP of 52.184.164.73, which is strange since all my users are, and connect from, Algeria. According to the ispinfo website (http://www.ispinfo.net/isp/52.184.164.73.html) this IP belongs to Microsoft. Could it be that the outlook app uses microsoft's servers to fetch the mail before handing them to the user ? Daniel.

2 2