April 2018 - dovecot - dovecot.org

RE: Re: Filesystem level backups?
by DurgaPrasad - DatasoftComnet 20 Apr '18

20 Apr '18

Hello all, Does doveadm backup - backup the attachments as well when using SIS ? Regards Durga Prasad +919849111010 -----Original Message----- From: dovecot [mailto:dovecot-bounces@dovecot.org] On Behalf Of Marc Stürmer Sent: 09 April 2018 18:05 To: Steve Atkins Cc: dovecot(a)dovecot.org Subject: {Spam?} Re: Filesystem level backups? Am 2018-04-09 00:55, schrieb Steve Atkins: > Will a simple backup of all the files of an multi-dbox give me a > consistent state when I recover, or do I need to do something (e.g. > lock writes to the mailbox) while I run a backup? If you are doing this backup with a fully operational mail storage: no, it won't. The reason is quite simple: mdbox saves the data of an email in several files, so there's a good chance that those files might slightly change during backup, while with Maildir all necessary data for the restore is stored in the file names. If you want to do a consistent backup, you do need file system snapshots and should make sure that this snapshot is consistent or to turn off the mail store during backup completely. But just doing rsync on the fly as with Maildir is just begging on your side to run into problems sooner or later... -- Greetings, Marc --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus

3 4

massive mail move
by David Mehler 20 Apr '18

20 Apr '18

Hello, I've got dovecot 2.3 going and a mailbox with a lot of system messages in it. What I'm wanting to do is massively move them to another mailbox. Is this doable with sieve? Thanks. Dave.

2 2

maildir vs dbox?
by David Mehler 20 Apr '18

20 Apr '18

Hello, I've got a Dovecot virtual email setup using maildir as the mailbox format. I am wondering from others who have chosen dbox a comparison between maildir and dbox or those who have migrated from one format to another. I am wondering if changing to dbox would be beneficial? Thanks. Dave.

4 3

Replication between servers with different mail_home path
by Arie Peterson 19 Apr '18

19 Apr '18

Hi, I have two dovecot servers that are configured to be replicated using dsync over ssh. This has been working well for a long time. Recently I changed the mail_home path on one of the servers, because of mail data migration. The old file system is still available, but has been made read-only for now – I plan to remove it once everything runs smoothly again. Now it seems replication is failing: the mail.err log file on both servers has lots of error messages saying that it tried to open a file on the old location and failed because it is read-only. Is this setup, with different mail_home paths on the two servers being replicated, even supported? If so, do I need to specify this explicitly in the configuration? I couldn't find relevant settings in the documentation.

1 0

Comparing Indexing Speed + ZFS question
by Qmail 18 Apr '18

18 Apr '18

Hi, have two quick questions: I am re-indexing some large mailboxes and I wanted to get some opinions on these speeds. I ran a random sort command on an unindexed mailbox in order to get it to index. Results, for a mailbox with 10GB Inbox, spread in 65 000 files: 3 UID SORT (ARRIVAL) US-ASCII OR OR OR FROM "dfgfgfgf" SUBJECT "dfgfgfgf" TO "dfgfgfgf" CC "dfgfgfgf" a03 select INBOX * SORT 3 OK Sort completed (2455.549 + 0.122 + 1.477 secs). Index cache file at the end was 31MB in size: 30756376 Apr 18 02:14 dovecot.index.cache Is that normal, slow or fast ? Second question is, how can I speed this up ? I store messages using Director over NFS , on top of ZFS. From what I've read it would be fairly safe to disable mail_fsync but I don't think that would help much with the indexing speed. Any other tips ?

2 2

Postfix sendmail cannot be called from Sieve redirect
by Ulrich Zehl 17 Apr '18

17 Apr '18

Sending (in my case: forwarding) messages from Sieve via sendmail does not work with version 2.3.1. I have narrowed it down to this simple test case: # dovecot -n # 2.3.1 (c5a5c0c82): /etc/dovecot/dovecot.conf # OS: Linux 4.4.0-119-generic x86_64 Ubuntu 16.04.4 LTS # Hostname: tuxi.topfen.net first_valid_gid = 200 first_valid_uid = 200 mail_location = mbox:~/mail passdb { args = password=secret driver = static } plugin { sieve = file:~/sieve;active=~/.dovecot.sieve } protocols = lmtp service lmtp { inet_listener lmtp { address = 127.0.0.1 port = 24 } } userdb { args = uid=200 gid=200 home=/home/vmail driver = static } protocol lmtp { mail_plugins = sieve } # cat /home/vmail/.dovecot.sieve redirect "test(a)example.invalid"; stop; $ swaks --protocol lmtp -s 127.0.0.1 -f ulrich(a)topfen.net -t ulrich(a)topfen.net === Trying 127.0.0.1:24... === Connected to 127.0.0.1. <- 220 tuxi.topfen.net Dovecot ready. -> LHLO tuxi.topfen.net <- 250-tuxi.topfen.net <- 250-8BITMIME <- 250-CHUNKING <- 250-ENHANCEDSTATUSCODES <- 250-PIPELINING <- 250 VRFY -> MAIL FROM:<ulrich(a)topfen.net> <- 250 2.1.0 OK -> RCPT TO:<ulrich(a)topfen.net> <- 250 2.1.5 OK -> DATA <- 354 OK -> Date: Tue, 17 Apr 2018 18:44:42 +0200 -> To: ulrich(a)topfen.net -> From: ulrich(a)topfen.net -> Subject: test Tue, 17 Apr 2018 18:44:42 +0200 -> X-Mailer: swaks v20130209.0 jetmore.org/john/code/swaks/ -> -> This is a test mailing -> -> . <** 451 4.3.0 <ulrich(a)topfen.net> Temporary internal error -> QUIT <- 221 2.0.0 Bye === Connection closed with remote host. $ tail -n 10 /var/log/mail.log Apr 17 18:45:36 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/421882.22643: Permission denied Apr 17 18:45:36 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/421882.22643: Permission denied Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: program `/usr/sbin/sendmail' was forcibly terminated with signal 15 Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: sieve: msgid=unspecified: failed to redirect message to <test(a)test.at>: Failed to execute sendmail (temporary failure) Apr 17 18:45:36 tuxi dovecot: lmtp(ulrich(a)topfen.net)<22639><dtJZGJIk1lpvWAAAcdhj4Q>: Error: sieve: Execution of script /home/vmail/.dovecot.sieve was aborted due to temporary failure (user logfile /home/vmail/.dovecot.sieve.log may reveal additional details) Apr 17 18:45:36 tuxi dovecot: lmtp(22639): Disconnect from 127.0.0.1: Client has quit the connection (state = READY) Apr 17 18:45:46 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/422091.22643: Permission denied Apr 17 18:45:46 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/422091.22643: Permission denied Apr 17 18:45:56 tuxi postfix/postdrop[22643]: warning: mail_queue_enter: create file maildrop/422227.22643: Permission denied Apr 17 18:45:56 tuxi dovecot: lmtp: Error: postdrop: warning: mail_queue_enter: create file maildrop/422227.22643: Permission denied As far as I can tell, this is because Dovecot somehow does not allow suid programs to run (Postfix' postdrop(1), which is invoked by sendmail(1), requires set-gid to write the queue file). What can I do to change this (besides using SMTP from Dovecot directly)? I am using Ubuntu 16.04 and the Dovecot packages from the community repository: ~$ dpkg -l dovecot* | grep ^ii ii dovecot-core 2:2.3.1-1 amd64 secure POP3/IMAP server - core files ii dovecot-lmtpd 2:2.3.1-1 amd64 secure POP3/IMAP server - LMTP server ii dovecot-sieve 2:2.3.1-1 amd64 secure POP3/IMAP server - Sieve filters support Ulrich

2 2

dovecot-lda "-f" argment optional?
by Sebastian Arcus 17 Apr '18

17 Apr '18

I use dovecot-lda to deliver incoming email from Exim to Dovecot. After having some problems with email bounces with empty envelope sender not being delivered because of an empty -f argument, I discovered that Exim can pass email to Dovecot using dovecot-lda without using the -f argument at all. Would this have any adverse effects anywhere else? The docs don't even seem to specify if -f is optional or not - but I guess it is - as it seems to be working without it. Does anybody know what is its role? I'd like to understand things a bit before I just start removing it permanently from my setup.

1 0

[bug] Auth cache, proxy and ITERINDEX
by azurit＠pobox.sk 17 Apr '18

17 Apr '18

Hi, today we have exceprienced really major difficulties with our proxy backend servers. Everything started after enabling auth cache: auth_cache_size = 100M auth_cache_verify_password_with_worker = yes auth_cache_ttl = 1 hour auth_cache_negative_ttl = 1 hour Soon after we start receiving lots of calls from our customers about 'missing all e-mail messages'. This was in errors logs on master server: Apr 16 14:37:10 server00 dovecot: imap(<censored_email_address>): Error: autoexpunge: Couldn't create dovecot.autoexpunge.lock lock: file_create_locked(/var/mail/vhosts/<censored_domain>/<censored_name>/home/dovecot.autoexpunge.lock) failed: safe_mkstemp(/var/mail/vhosts/<censored_domain>/<censored_name>/home/dovecot.autoexpunge.lock) failed: No such file or directory Looks like that all users, who are placed on proxy backends, were logged correctly BUT not proxied to the right server so they saw empty mailboxes (Dovecot also created some directories on master server). What is worse, their email software deleted all local emails so they are now downloading everything from scratch (100Gs of mails so it will take some time). After auth cache was disabled, things started to go to normal EXCEPT users were reporting they don't see all folders - which was true, only few on them were visible (INBOX, trash, sent etc.). We tried everything to make them visible again, for example deleting indexes, but nothing helped. Finally, removing option ITERINDEX from mail_location helped. Can anyone explain what happened? Thank you azurIt

2 2

spamc scripts in IMAPSieve docs.
by Reio Remma 17 Apr '18

17 Apr '18

Hello! I noticed SpamAssassin *spamc* usage has entered the documentation at https://wiki2.dovecot.org/HowTo/AntispamWithSieve I'm wondering if the -C (report) option in sa-learn-ham.sh should use revoke instead of report for --ham messages? I started using imapsieve with spamc myself just a few weeks ago, but I haven't used the reporting ability yet. Reio

2 4

File system permissions - setgid bit and Netapp NFS volumes
by Shaun Johnson 17 Apr '18

17 Apr '18

Greetings Dovecot List, I have a bit of an edge case I am trying to resolve. I am currently using dovecot on Ubuntu 14.04 - Ubuntu package version: 1:2.2.9-1ubuntu2.3 I have attached the output of doveconf -n to this email - but to describe the configuration in a nutshell: my server is configured to use Maildir storage I do not use dovecot delivery service (there is a separate program in my system delivering messages) I am using a 'checkpassword' mechanism for authentication My mail storage is mounted via NFSv3 from a Netapp Filer FAS3050 configured to set anonuid=89, read write access, and root access. Dovecot has been configured with: mail_fsync = always mail_nfs_index = yes mail_nfs_storage = yes mmap_disable = yes lock_method = fcntl In order to function with the other components of my system, the Mail Storage directories are expected to have permissions: drwx--S--- uid 89: gid 89 The oddity I am encountering came up recently when I set up a job to monitor and correct directory permissions. I found after observing the system in action that overtime, the ~/Maildir directories for all active mailboxes would mysteriously lose the setgid bit. Now, while the system still functioned, it was something unexpected so I investigated further by turning on auditd to monitor what was changing the permissions on the path. After tracing through the results, I found an entry indicating that the 'imap' process of dovecot was performing a 'chown' call on the directory, with an end result of the previous mode 2700 being changed to 0700. Now I understand that on some filesystems, a 'chown' call can end up losing any setuid or setgid bits on an inode - and apparently Netapp's WAFL file system is one of them. Looking at the dovecot source code, I *believe* the source of the chown call is in src/lib/nfs-workarounds.c function nfs_flush_chown_uid - where a 'chown' call is being used to flush NFS attribute caches? for what I believe is the purposes of ascertaining file lock status? My question for you folks is if anyone else has encountered this on other file systems (or filers) - and if there is a native method in dovecot - either the version I am currently running or an updated newer version - that allows one to either retain the permissions assigned to a directory or specify the permissions I expect on the Maildir directory structures. Any feedback would be welcome, thanks! -- Shaun Johnson LinuxMagic Inc. 604-682-0300 Beautiful British Columbia, Canada

2 2