- dovecot - dovecot.org

"unknown user - trying the next userdb" Info in log
by James Brown 30 Jan '19

30 Jan '19

I’ve set up a new mail server. Auth-worker is giving me this error all the time: auth-worker(11160): Info: passwd(sales(a)bordo.com.au <mailto:sales@bordo.com.au>,10.0.0.54,<YSFoAYHAGoLAqAEo>): unknown user - trying the next userdb Next line is from imap-login with successful login. Using MySQL as user database. Auth-sql.comf.ext is: passdb { driver = sql # Path for SQL configuration file, see example-config/dovecot-sql.conf.ext args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } # "prefetch" user database means that the passdb already provided the # needed information and there's no need to do a separate userdb lookup. # <doc/wiki/UserDatabase.Prefetch.txt> userdb { driver = prefetch } userdb { driver = sql args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } # If you don't have any user-specific settings, you can avoid the user_query # by using userdb static instead of userdb sql, for example: # <doc/wiki/UserDatabase.Static.txt> #userdb { #driver = static #args = uid=vmail gid=vmail home=/var/vmail/%u #} Any ideas why it is gives me this error before logging in? Where should I look? Thanks, James.

3 5

doveadm batch crash
by Jonathan Casiot 30 Jan '19

30 Jan '19

Hi I'm having an issue with doveadm batch - the following command always crashes: doveadm batch : mailbox status all inbox Whereas the following work as expected: doveadm mailbox status all inbox doveadm batch : mailbox list Any assistance gratefully received. Debug output from the crash: Debug: Loading modules from directory: /usr/lib64/dovecot Debug: Module loaded: /usr/lib64/dovecot/lib01_acl_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib10_mail_crypt_plugin.so Debug: Module loaded: /usr/lib64/dovecot/lib90_stats_plugin.so Debug: Loading modules from directory: /usr/lib64/dovecot/doveadm Debug: Module loaded: /usr/lib64/dovecot/doveadm/lib10_doveadm_acl_plugin.so Debug: Skipping module doveadm_expire_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_expire_plugin.so: undefined symbol: expire_set_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_quota_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib10_doveadm_quota_plugin.so: undefined symbol: quota_user_module (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_lucene_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_lucene_plugin.so: undefined symbol: lucene_index_iter_deinit (this is usually intentional, so just ignore this message) Debug: Skipping module doveadm_fts_plugin, because dlopen() failed: /usr/lib64/dovecot/doveadm/lib20_doveadm_fts_plugin.so: undefined symbol: fts_user_get_language_list (this is usually intentional, so just ignore this message) Debug: Module loaded: /usr/lib64/dovecot/doveadm/libdoveadm_mail_crypt_plugin.so doveadm(jc): Debug: Effective uid=1001, gid=100, home=/home/jc doveadm(jc): Debug: acl: No acl_shared_dict setting - shared mailbox listing is disabled doveadm(jc): Debug: mail_crypt_plugin: mail_crypt_curve setting missing - generating EC keys disabled doveadm(jc): Debug: Namespace inbox: type=private, prefix=, sep=., inbox=yes, hidden=no, list=yes, subscriptions=yes location=mdbox:~/mdbox:ALT=~/Archive/tape/mdbox doveadm(jc): Debug: fs: root=/home/jc/mdbox, index=, indexpvt=, control=, inbox=, alt=/home/jc/Archive/tape/mdbox doveadm(jc): Debug: acl: initializing backend with data: vfile doveadm(jc): Debug: acl: acl username = jc doveadm(jc): Debug: acl: owner = 1 doveadm(jc): Debug: acl vfile: Global ACLs disabled doveadm(jc): Panic: file mail-storage.c: line 875 (mailbox_set_reason): assertion failed: (reason != NULL) doveadm(jc): Error: Raw backtrace: /usr/lib64/dovecot/libdovecot.so.0(+0xa0e9e) [0x7fc993023e9e] -> /usr/lib64/dovecot/libdovecot.so.0(default_fatal_handler+0x2a) [0x7fc993023f0a] -> /usr/lib64/dovecot/libdovecot.so.0(i_fatal+0) [0x7fc992fb3c97] -> /usr/lib64/dovecot/libdovecot-storage.so.0(+0x49856) [0x7fc993301856] -> /usr/bin/doveadm(+0x348e2) [0x5631dbb5f8e2] -> /usr/bin/doveadm(+0x2e1ad) [0x5631dbb591ad] -> /usr/bin/doveadm(+0x2bd2c) [0x5631dbb56d2c] -> /usr/bin/doveadm(+0x2c92a) [0x5631dbb5792a] -> /usr/bin/doveadm(doveadm_mail_try_run+0x215) [0x5631dbb58285] -> /usr/bin/doveadm(main+0x46a) [0x5631dbb473fa] -> /lib64/libc.so.6(__libc_start_main+0xf5) [0x7fc992bd83d5] -> /usr/bin/doveadm(+0x1c565) [0x5631dbb47565] Aborted (core dumped) And doveconf -n output: # 2.2.36 (1f10bfa63): /etc/dovecot/dovecot.conf # OS: Linux 3.10.0-957.1.3.el7.x86_64 x86_64 CentOS Linux release 7.6.1810 (Core) # Hostname: ### redacted ### first_valid_gid = 100 first_valid_uid = 1000 last_valid_gid = 100 last_valid_uid = 1999 listen = * login_greeting = IMAP ready login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e session=<%{session}> %k mail_attribute_dict = file:%h/mdbox/dovecot-attributes mail_location = mdbox:~/mdbox:ALT=~/Archive/tape/mdbox mail_plugins = acl stats mail_crypt mailbox_list_index = yes mdbox_rotate_size = 16 M namespace inbox { inbox = yes location = mailbox Drafts { special_use = \Drafts } mailbox Junk { special_use = \Junk } mailbox Sent { special_use = \Sent } mailbox "Sent Messages" { special_use = \Sent } mailbox Trash { special_use = \Trash } prefix = separator = . } passdb { driver = pam } plugin { acl = vfile mail_crypt_global_private_key = <### redacted ### mail_crypt_global_public_key = <### redacted ### mail_crypt_save_version = 2 stats_memory_limit = 16 M stats_refresh = 30 secs stats_track_cmds = yes } postmaster_address = ### redacted ### protocols = imap service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } } service imap-login { process_limit = 200 process_min_avail = 4 service_count = 1 } service stats { fifo_listener stats-mail { group = users mode = 01224 } } ssl = required ssl_cert = <### redacted ### ssl_key = # hidden, use -P to show it ssl_prefer_server_ciphers = yes ssl_protocols = !SSLv2 !SSLv3 !TLSv1 userdb { driver = passwd } verbose_proctitle = yes protocol imap { imap_idle_notify_interval = 10 mins imap_metadata = yes mail_max_userip_connections = 20 mail_plugins = acl stats mail_crypt imap_acl imap_stats } Regards, -- Jonathan

2 2

send and receive error message
by Chandran Manikandan 29 Jan '19

29 Jan '19

Hi Folks, I am running qmailtoaster on COS6 32 bit. Send and receive error message coming frequently and inbox emails are not downloading in outlook 2016 of office 365 email client. But if i restart the dovecot service then start downloading the emails in inbox then send and receive error message gone. Outlook 2010 doesn't have the same issue. Is this server issue? or email client version issues? or if i need to use standalone outlook product instead office 365? I do the dovecot service restart every one hour for this issue . if possible to do the cron job for dovecot service restart for every one hour daily or any other solution there. Kindly Any one help me. -- *Thanks,* *Manikandan.C* *System Administrator*

1 0

Re: Rsync to backup dbox with SIS (Daniel Miller)
by ash-dovecot＠comtek.co.uk 28 Jan '19

28 Jan '19

> Probably not the answer you want - but I would strongly suggest using > Dovecot replication.? Dovecot replication Just Works - so don't reinvent > the wheel when Timo provided such a polished tool already.? Okay, thanks, I'll look at replication for the warm spare. Although we do use shared folders, which your link below says don't work properly. > And based on > my previous SIS experience - while dbox is nice I would suggest avoiding > SIS until there are reports of more development.? Sdbox will be solid - > which is what you want for an archive - though maildir would be the > safest.? Archives don't need to be rapid-access - they need to be > dependable.? SIS is wonderful for space saving - but until there's more > safety checks built-in I'd suggest avoiding it for production backups.? > Drives are cheap - lost data, lost time, lost hair, lost sanity...is not. Our main motivation for dbox is SIS. We don't have a very high volume of mail, so maildir speed is unlikely to be an issue. What we do have is a lot of users who like to mail around large files. Drives are cheap, but there are advantages to a smaller mailstore. That said, data security wins. We will probably abandon dbox entirely and continue with maildir, thanks. > The backups don't have to be "visible" to other machines - don't even > have to be running IMAP/POP services (I think).? And the replication > command is run - via ssh (https://wiki.dovecot.org/Replication) - so > what's the problem? Well, our backup servers are supposed to be running a minimalist set of software. If we need to restore from backup after a security issue or application bug on a production machine, then we are trying to minimise the chance that the backup machine itself could be affected. Anyway, if we aren't using SIS, then we needn't use dbox, and therefore backup machines can continue to run a minimalist OS with only rsync. Thanks, Ash --- To opt-in to receiving communications from us or to update your email preferences please go to https://lists.comtek.co.uk/prefs. To be removed from receiving all communications from the Comtek Group, please unsubscribe at https://lists.comtek.co.uk/prefs and accept our apologies for any inconvenience caused. The Comtek Group respects your privacy. Please find our privacy policy at http://www.comtek.co.uk/privacy-policy This email and any attachments are confidential. If you are not the named recipient, please notify the sender immediately. The unauthorised use, disclosure, copying or alteration of this message or its attachments is strictly prohibited. Opinions expressed in this message are those of the individual and not Comtek Network Systems (UK) Ltd unless indicated by an authorised representative independent of this message. Comtek Network Systems does not accept responsibility for any changes to this message after it has been sent. Company Details: Comtek Network Systems (UK) Ltd. Registered Office: Comtek Network Systems (UK) Ltd, Unit 108, Tenth Avenue (Zone 3), Deeside Industrial Park, Deeside, Flintshire, CH5 2UA. Comtek Network Systems (UK) Ltd is a company registered in England and Wales with company number 05929974.

1 0

all of a sudden, slow resync
by Victor Sudakov 27 Jan '19

27 Jan '19

Dear Colleagues, All of a sudden, I've begun receiving this output when reindexing my mail boxes: # /usr/local/bin/doveadm force-resync -u vas "*" doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/sent/dovecot.index.log was locked for 55 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-30-06-2010/dovecot.index.log was locked for 56 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-2014/dovecot.index.log was locked for 95 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-20-06-2012/dovecot.index.log was locked for 72 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-18-05-2009/dovecot.index.log was locked for 31 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dss-07-12-2008/dovecot.index.log was locked for 56 seconds (Mailbox was synchronized) doveadm(vas): Warning: Transaction log file /home/vas/Mail/sibptus/.imap/dns-operations/dovecot.index.log was locked for 95 seconds (Mailbox was synchronized) And it really takes forever to finish. What could cause this? FreeBSD 11.2-RELEASE-p8, dovecot-2.3.3_4 -- Victor Sudakov, VAS4-RIPE, VAS47-RIPN 2:5005/49@fidonet http://vas.tomsk.ru/

1 0

auth-worker: Error: double free or corruption (fasttop)
by Marcel Lorenz 27 Jan '19

27 Jan '19

Dear list, since the distri upgrade to glibc 2.28, i can't build working dovecot binary's. Older with glibc 2.27 are working. The distri is LFS based. MariaDB 10.2.18 and 10.3.11 makes no difference. Having this on different systems. Messages Log: Dec 19 17:10:38 test kernel: traps: auth[30189] general protection ip:7f4b96d6598c sp:7ffef1f31710 error:0 in libmariadb.so.3[7f4b96d4e000+67000] Dec 19 17:11:39 test kernel: auth[30299]: segfault at 560c57366c40 ip 0000560c57366c40 sp 00007fffe5b51f18 error 15 Dec 19 17:12:39 test kernel: traps: auth[30346] general protection ip:7f55601f298c sp:7ffe597929c0 error:0 in libmariadb.so.3[7f55601db000+67000] Dec 19 17:13:39 test kernel: traps: auth[30392] general protection ip:7fcd2317298c sp:7ffecd6f4cb0 error:0 in libmariadb.so.3[7fcd2315b000+67000] Dovecot log: 2018-12-19 17:07:37 auth-worker: Fatal: master: service(auth-worker): child 30053 killed with signal 11 (core dumped) 2018-12-19 17:09:38 auth-worker: Error: double free or corruption (fasttop) 2018-12-19 17:09:38 auth-worker: Fatal: master: service(auth-worker): child 30098 killed with signal 6 (core dumped) 2018-12-19 17:10:38 auth-worker: Fatal: master: service(auth-worker): child 30189 killed with signal 11 (core dumped) Output of dovecot -n: # 2.3.4 (0ecbaf23d): /etc/dovecot/dovecot.conf # Pigeonhole version 0.5.4 (60b0f48d) # OS: Linux 4.14.80 x86_64 # Hostname: test.testing.local auth_mechanisms = plain login auth_username_chars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ01234567890.-_@+*#" base_dir = /var/run/dovecot/ debug_log_path = /var/log/dovecot/debug.log dict { expire = mysql:/etc/dovecot/dovecot-dict-sql-expire.conf sqldomainquota = mysql:/etc/dovecot/dovecot-dict-sql-domain.conf sqluserquota = mysql:/etc/dovecot/dovecot-dict-sql-user.conf } info_log_path = /var/log/dovecot/info.log lda_mailbox_autocreate = yes lda_mailbox_autosubscribe = yes listen = * log_path = /var/log/dovecot/main.log log_timestamp = "%Y-%m-%d %H:%M:%S " login_greeting = dovecot ready login_log_format = %s login_log_format_elements = user=<%u> method=%m rip=%r lip=%l mpid=%e %c %k mail_location = mdbox:/var/vmail/%d/%n/ mail_max_userip_connections = 4 mail_plugins = " zlib quota expire acl fts fts_lucene" mailbox_idle_check_interval = 1 mins maildir_very_dirty_syncs = yes managesieve_notify_capability = mailto managesieve_sieve_capability = fileinto reject envelope encoded-character vacation subaddress comparator-i;ascii-numeric relational regex imap4flags copy include variables body enotify environment mailbox date index ihave duplicate mime foreverypart extracttext namespace { list = children location = maildir:/var/vmail/%d/public:INDEXPVT=/var/vmail/%d/public prefix = Public. separator = . subscriptions = no type = public } namespace { list = children location = maildir:/var/vmail/%d/%%u:CONTROL=/var/vmail/%d/shared/%%u:INDEX=/var/vmail/$ prefix = shared.%%n. separator = . subscriptions = yes type = shared } namespace inbox { inbox = yes location = mailbox Archive { auto = subscribe special_use = \Archive } mailbox Drafts { auto = subscribe special_use = \Drafts } mailbox Junk { auto = subscribe special_use = \Junk } mailbox Sent { auto = subscribe special_use = \Sent } mailbox Trash { auto = subscribe special_use = \Trash } prefix = separator = . } passdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } plugin { acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes al = vfile expire = Trash expire2 = Trash/* expire3 = Spam expire_cache = yes expire_dict = proxy::expire fts = lucene fts_lucene = whitespace_chars=@.-+_*:/\"' last_login_key = # hidden, use -P to show it quota = dict:User Quota::proxy::sqluserquota quota_exceeded_message = Quota exceeded quota_grace = 10%% quota_rule2 = Trash:storage=+15%% quota_rule3 = Spam:ignore quota_status_nouser = DUNNO quota_status_overquota = 552 5.2.2 Mailbox is over quota / Postfach ist voll quota_status_success = DUNNO quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u sieve = /var/vmail/%d/%n/sieve/active-script.sieve sieve_before = /var/vmail/sieve/spam-global.sieve sieve_dir = /var/vmail/%d/%n/sieve/scripts/ sieve_global = /var/lib/dovecot/sieve/global/ sieve_max_actions = 32 sieve_max_redirects = 4 sieve_max_script_size = 1M trash = /etc/dovecot/dovecot-trash.conf zlib_save = gz zlib_save_level = 6 } postmaster_address = postmaster(a)testing.local protocols = imap pop3 lmtp sieve service auth-worker { user = doveauth } service auth { unix_listener /var/spool/postfix/private/auth { group = postfix mode = 0660 user = postfix } unix_listener auth-userdb { mode = 0660 user = doveauth } } service dict { unix_listener dict { mode = 0600 user = vmail } } service imap-login { inet_listener imap { port = 143 } inet_listener imaps { port = 993 ssl = yes } process_min_avail = 0 } service lmtp { unix_listener /var/spool/postfix/private/dovecot-lmtp { group = postfix mode = 0660 user = postfix } } service managesieve-login { inet_listener sieve { port = 4190 } service_count = 1 } service pop3-login { inet_listener pop3 { port = 110 } inet_listener pop3s { port = 995 ssl = yes } } service quota-status { client_limit = 1 executable = quota-status -p postfix inet_listener { port = 10022 } } service quota-warning { executable = script /usr/local/bin/quota-warning.sh user = vmail } ssl_alt_cert = </etc/ssl/certs/postfix-rsa.crt ssl_alt_key = # hidden, use -P to show it ssl_cert = </etc/ssl/certs/postfix-ecc.crt ssl_dh = # hidden, use -P to show it ssl_key = # hidden, use -P to show it ssl_min_protocol = TLSv1.2 ssl_prefer_server_ciphers = yes userdb { args = /etc/dovecot/dovecot-sql.conf driver = sql } verbose_proctitle = yes protocol imap { imap_logout_format = in=%i out=%o imap_max_line_length = 64 k mail_max_userip_connections = 10 mail_plugins = " zlib quota expire acl fts fts_lucene acl imap_acl imap_quota mail_log last_login notify imap_zlib trash" } protocol pop3 { mail_max_userip_connections = 4 mail_plugins = " zlib quota expire acl fts fts_lucene last_login expire" pop3_client_workarounds = outlook-no-nuls oe-ns-eoh pop3_enable_last = no pop3_lock_session = no pop3_logout_format = retr=%r/%b, del=%d/%m, size=%s pop3_no_flag_updates = yes pop3_reuse_xuidl = no pop3_save_uidl = yes pop3_uidl_format = %08Xu%08Xv } protocol lmtp { mail_plugins = " zlib quota expire acl fts fts_lucene sieve" postmaster_address = postmaster(a)test.local } protocol lda { auth_socket_path = /var/run/dovecot/auth-master info_log_path = /var/log/dovecot/deliver.log log_path = /var/log/dovecot/deliver.log mail_fsync = optimized mail_plugins = " zlib quota expire acl fts fts_lucene trash autocreate sieve" postmaster_address = postmaster(a)testing.local quota_full_tempfail = yes rejection_reason = Your message to <%t> with subject of <%s> was automatically rejected:%n%r rejection_subject = Mailbox Full: %s } protocol sieve { mail_max_userip_connections = 10 managesieve_implementation_string = Dovecot Pigeonhole managesieve_logout_format = bytes=%i/%o managesieve_max_compile_errors = 5 managesieve_max_line_length = 65536 } Thanks in advance! Marcel

3 3

download Dovecot emails over rsync (done!), possible alternatives
by Carles Pina i Estany 27 Jan '19

27 Jan '19

Hello, Apologies for the extra long email. It's a situation with a solved problem and I might need to solve it again, I'd like to have some feedback. Two years ago, in a trip around the Antarctic with a very limited communication I deployed a mail system based on Postfix+Dovecot. I need to do something similar again and I used a "hack" that I thought that I might avoid (that time, let's say, everything was last minute... or later than last minute!). I installed Dovecot on the internet server and the server on the ship. On the ship I had a fetchmail downloading "small emails" (up to 100 KB if I remember correctly) and delivering to the local server. For bigger emails fetchmail had a "size limit" and fetchmail was not downloading them. I wrote a Python script that was connecting from the ship, to the internet server, and was listing the emails that were "too big to be downloaded" (this was for each user. I could have done something different with procmailrc I think on the receiving time). The Python script was fetching, if I remember correctly, the email ID, date, subject and size. The Python script was sending an email to the ship server's user saying "there is an email for you with this size, from, date and subject and this is the email ID" (and inserting this into a database to avoid sending the same notification to the users again). Then the scientists sometimes were ignoring the email, were asking to the sender to send a smaller version, or asking me to download it when possible. If they wanted me to download the email: I had a script that was connecting to the internet server, opening /home/$USER/Maildir/dovecot-uidlist, finding the file for that email UUID (this was a simple script on the internet server and I was executing it using ssh and getting the output). Then it was creating a new IMAP folder named "Downloaded" in the ship's server (it was just creating the folder in the file system and adding it in the subscriptions file, all using file system operations from Python). Then using rsync it was downloading the email file from the internet's server to the ship's server temporary file (taking whatever it would take, with many retries, etc.) and then moving it to the Downloaded folder. (note: it was using rsync and not IMAP because the connection was slow and very unstable. With rsync and the correct options it was continuing the download and retrying until success). For the next expedition I'll have more time to prepare for a similar setup. I've considered if I should stop relying on dovecot-uidlist, local file system's, subscriptions, etc. and if I should write a plugin for Dovecot to expose part of this information using... a JSON API? IMAP commands? The local ship's server bit: could I inject a SMTP mail file (downloaded) using Dovecot admin commands? some API? Any advantage? To discover which file to download using rsync: instead of opening the dovecot-uidlist any better method? Any other different approach that I might want to consider? Sorry again for the long message. I thought that this might need some context. Also, thanks to the design of Dovecot I could do what I did in a very simple way. Actually Dovecot design was as I would have designed for this use case (the index file, subscription from the mailing list, etc.). Thank you very much again! and thanks for the great Dovecot as well! -- Carles Pina i Estany Web: http://pinux.info || Blog: http://pintant.cat GPG Key 0x8CD5C157

1 1

Rsync to backup dbox with SIS
by ash-dovecot＠comtek.co.uk 26 Jan '19

26 Jan '19

We will be deploying a replacement Dovecot server soon, and we are planning to use maildir for the primary storage, but with an archive namespace using mdbox (or perhaps sdbox), and SIS. Our backup servers and (luke)warm spare server need to obtain full copies of the mail store. For the maildirs I know I can simply use rsync (we already use it here). I'm a little wary of using rsync with mdbox and SIS though. Obviously we would have to use the -H option of rsync to avoid duplicating the hard linked data, does anybody have any experience with how this scales? I've considered simply rsyncing the /var/mail/dovecot/attach/??/??/hashes folders. Is there a tool that can reconstruct the /var/mail/dovecot/attach/??/??/*-* links for a list of messages in that case? I'm aware of "doveadm backup", and (although it currently throws up a few errors) it seems like it might be a valid solution for our warm spare server. Our backup servers, on the other hand, aren't supposed to be visible to the production machines, with the exception of the backup machine sshing in to do an rsync each night. We can't install dovecot on them. To perform partial email recovery in maildir, we can simply log into a backup server, grep for emails of interest, and then scp them into a maildir new/ subfolder on the mail server to recover them. What about using dbox? I can identify /var/mail/dovecot/archive/ash/dbox/storage/m.* files, and identify necessary attachments under /var/mail/dovecot/attach/??/??/ but once I've copied them over to the mail server is there a safe way to import them? Actually, converting to a maildir style file would be adequate if there is a script anywhere to do this. We'd still rather be backing up the deduplicated mailstore for obvious reasons. Has anybody got any suggestions? I'm aware of "dsync backup questions" ( https://www.dovecot.org/list/dovecot/2013-February/088070.html ) but there didn't really seem to be a solution. Thanks, Ash PS Our current server has 257 users (though perhaps 100 are active), using around 225G in total. PPS Proposed new server configuration: mail_location = maildir:~/Maildir:LAYOUT=fs mail_attachment_dir = /var/mail/dovecot/attach mail_attachment_fs = sis posix mail_attachment_hash = %{sha1} mail_attachment_min_size = 128 k namespace _archive { list = yes location = mdbox:/var/mail/dovecot/archive/%n/dbox prefix = Archive/ separator = / subscriptions = yes type = private } namespace _private { inbox = yes type = private separator = / subscriptions = yes prefix = } --- To opt-in to receiving communications from us or to update your email preferences please go to https://lists.comtek.co.uk/prefs. To be removed from receiving all communications from the Comtek Group, please unsubscribe at https://lists.comtek.co.uk/prefs and accept our apologies for any inconvenience caused. The Comtek Group respects your privacy. Please find our privacy policy at http://www.comtek.co.uk/privacy-policy This email and any attachments are confidential. If you are not the named recipient, please notify the sender immediately. The unauthorised use, disclosure, copying or alteration of this message or its attachments is strictly prohibited. Opinions expressed in this message are those of the individual and not Comtek Network Systems (UK) Ltd unless indicated by an authorised representative independent of this message. Comtek Network Systems does not accept responsibility for any changes to this message after it has been sent. Company Details: Comtek Network Systems (UK) Ltd. Registered Office: Comtek Network Systems (UK) Ltd, Unit 108, Tenth Avenue (Zone 3), Deeside Industrial Park, Deeside, Flintshire, CH5 2UA. Comtek Network Systems (UK) Ltd is a company registered in England and Wales with company number 05929974.

2 1

problem in setting up proxy
by Ted 25 Jan '19

25 Jan '19

Hello, We're having difficulty with our updated cluster of dovecot servers accessing the email storage on the NFS mounts. It seems index files get corrupted when 2 backend mailservers access the same account, and from documentation setting up a director proxy in front of the backup servers. I'm trying to just set up a straight proxy first, which the documents say is the first step, and although I can see the connections coming into the server when I try to login via the proxy, the connection times out and there are no logs from dovecot anywhere saying what happened to the connection. The configs I have set up for this in dovecot are: dovecot.conf # Protocols we want to be serving. protocols = imap pop3 #when re-enabling quota enforcement add quota in below: mail_plugins = $mail_plugins mail_log notify protocol imap { # Space separated list of plugins to load (default is global mail_plugins). #when re-enabling quota enforcement add imap_quota in below: mail_plugins = $mail_plugins } # A comma separated list of IPs or hosts where to listen in for connections. # "*" listens in all IPv4 interfaces, "::" listens in all IPv6 interfaces. # If you want to specify non-default ports or anything more complex, # edit conf.d/master.conf. #listen = *, :: # Base directory where to store runtime data. #base_dir = /var/run/dovecot/ # Name of this instance. In multi-instance setup doveadm and other commands # can use -i <instance_name> to select which instance is used (an alternative # to -c <config_path>). The instance name is also added to Dovecot processes # in ps output. #instance_name = dovecot # Greeting message for clients. login_greeting = Welcome to easyMail. shutdown_clients = yes # Most of the actual configuration gets included below. The filenames are # first sorted by their ASCII value and parsed in that order. The 00-prefixes # in filenames are intended to make it easier to understand the ordering. !include conf.d/*.conf # A config file can also tried to be included without giving an error if # it's not found: !include_try local.conf service auth { unix_listener auth-master { mode = 0600 user = vmail } } conf.d/10-auth.conf ## ## Authentication processes ## # Username formatting before it's looked up from databases. You can use # the standard variables here, eg. %Lu would lowercase the username, %n would # drop away the domain if it was given, or "%n-AT-%d" would change the '@' into # "-AT-". This translation is done after auth_username_translation changes. auth_username_format = %Lu # Space separated list of wanted authentication mechanisms: # plain login digest-md5 cram-md5 ntlm rpa apop anonymous gssapi otp skey # gss-spnego # NOTE: See also disable_plaintext_auth setting. auth_mechanisms = plain login # # Password database is used to verify user's password (and nothing more). # You can have multiple passdbs and userdbs. This is useful if you want to # allow both system users (/etc/passwd) and virtual users to login without # duplicating the system users into virtual database. # # <doc/wiki/PasswordDatabase.txt> # # User database specifies where mails are located and what user/group IDs # own them. For single-UID configuration use "static" userdb. # # <doc/wiki/UserDatabase.txt> #!include auth-deny.conf.ext #!include auth-master.conf.ext #!include auth-system.conf.ext #!include auth-sql.conf.ext #!include auth-ldap.conf.ext #!include auth-passwdfile.conf.ext #!include auth-checkpassword.conf.ext #!include auth-vpopmail.conf.ext !include auth-static.conf.ext conf.d/auth-static.conf.ext # Static passdb. Included from auth.conf. # This can be used for situations where Dovecot doesn't need to verify the # username or the password, or if there is a single password for all users: # # - proxy frontend, where the backend verifies the password # - proxy backend, where the frontend already verified the password # - authentication with SSL certificates # - simple testing passdb static { driver = static args = nopassword=y default_fields = proxy=y host=10.5.10.121 } #passdb { # driver = static # args = password=test #} #userdb { # driver = static # args = uid=vmail gid=vmail home=/home/%u #} conf.d/10-logging.conf ## ## Log destination. ## # Log file to use for error messages. "syslog" logs to syslog, # /dev/stderr logs to stderr. #log_path = syslog # Log file to use for informational messages. Defaults to log_path. #info_log_path = # Log file to use for debug messages. Defaults to info_log_path. #debug_log_path = # Syslog facility to use if you're logging to syslog. Usually if you don't # want to use "mail", you'll use local0..local7. Also other standard # facilities are supported. #syslog_facility = mail ## ## Logging verbosity and debugging. ## # Log unsuccessful authentication attempts and the reasons why they failed. auth_verbose = yes # In case of password mismatches, log the attempted password. Valid values are # no, plain and sha1. sha1 can be useful for detecting brute force password # attempts vs. user simply trying the same password over and over again. # You can also truncate the value to n chars by appending ":n" (e.g. sha1:6). #auth_verbose_passwords = no # Even more verbose logging for debugging purposes. Shows for example SQL # queries. auth_debug = yes # In case of password mismatches, log the passwords and used scheme so the # problem can be debugged. Enabling this also enables auth_debug. #auth_debug_passwords = no # Enable mail process debugging. This can help you figure out why Dovecot # isn't finding your mails. mail_debug = yes # Show protocol level SSL errors. verbose_ssl = yes # mail_log plugin provides more event logging for mail processes. plugin { # Events to log. Also available: flag_change append #mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename # Available fields: uid, box, msgid, from, subject, size, vsize, flags # size and vsize are available only for expunge and copy events. #mail_log_fields = uid box msgid size } I'm basically expecting this to forward the login requests on to 10.5.10.121 when I try to access the email account through the proxy. When I attempt this I am able to see the connections in a tcp dump, but dovecot does not log anything about the attempt. I clearly must be missing something, can you let me know what I need to do or check? Thank you Ted easyDNS Technologies

4 6

Garbled mails in Maildir storage
by Florian Lohoff 25 Jan '19

25 Jan '19

Hi, from time to time i am having mails delivered to my anon maildrop which completely break mailbox listing. I ignored it mostly as this is an anon maildrop for one time emails so loosing spam wasnt an issue. Today i had a mailbox which wouldnt list at all which was caused by 50 consecutive mails beeing broken. (I am showing the last 20 in the web interface) After some debugging i found a bunch of mails like this: Return-Path: <*************************> X-Original-To: test(a)wasteland.rfc822.org Delivered-To: inbox(a)wasteland.rfc822.org Received: from node11.tmdvps.com (unknown [184.154.211.109]) by pax.zz.de (Postfix) with SMTP id 3B73DD00B5 for <test(a)wasteland.rfc822.org>; Thu, 8 Jun 2017 17:09:25 +0200 (CEST) <U+FEFF>Date: Thu, 8 Jun 2017 10:09:24 +0000 (GMT) From: Apple Support <*************************> REPLY-TO: do_not_reply(a)apple.com To: test(a)wasteland.rfc822.org [ ... ] Yes - there is an UTF-8 BOM in the middle of the mail header. Breaks parsing of the mail header of dovecot. No From and To in the listing. And i found even more: Return-Path: <**********************> X-Original-To: test(a)wasteland.rfc822.org Delivered-To: inbox(a)wasteland.rfc822.org Received: from 36.248.164.233 (unknown [36.248.164.233]) by pax.zz.de (Postfix) with SMTP id B2B0F1000FF; Tue, 22 Jan 2019 10:37:35 +0100 (CET) Received: from 6.51.138.64 by ; Tue, 22 Jan 2019 04:36:35 -0500 Message-ID: <B[20 Mails that end with garbage in them middle of the mail header. This is a standard Maildir storage using the dovecot transport via postfix. Does anyone have an explanation for these kind of mailstorage breakage? How could broken mails like this even be processed by postfix, dovecot deliver etc. As the last mail is pretty current i quickly grepped the postfix logs: Jan 22 10:37:35 pax postfix/smtpd[20269]: B2B0F1000FF: client=unknown[36.248.164.233] Jan 22 10:37:37 pax postfix/cleanup[20371]: B2B0F1000FF: message-id=<B[20 Jan 22 10:37:37 pax postfix/qmgr[9371]: B2B0F1000FF: from=<**********************>, size=233, nrcpt=2 (queue active) Jan 22 10:37:37 pax postfix/pipe[19192]: B2B0F1000FF: to=<inbox(a)wasteland.rfc822.org>, orig_to=<test108(a)wasteland.rfc822.org>, relay=wasteland, delay=2.2, delays=1.7/0/0/0.41, dsn=2.0.0, status=sent (delivered via wasteland service) Jan 22 10:37:37 pax postfix/pipe[21064]: B2B0F1000FF: to=<inbox(a)wasteland.rfc822.org>, orig_to=<test(a)wasteland.rfc822.org>, relay=wasteland, delay=2.2, delays=1.7/0.01/0/0.48, dsn=2.0.0, status=sent (delivered via wasteland service) Jan 22 10:37:37 pax postfix/qmgr[9371]: B2B0F1000FF: removed So even postfix found the message id to be truncated. Does dovecot have a "broken maildir mail storage mail dictionary" for automated tests? Flo -- Florian Lohoff f(a)zz.de UTF-8 Test: The 🐈 ran after a 🐁, but the 🐁 ran away

1 0